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JOINT  STRIKE  FIGHTER 

Restructuring  Places  Program  on  Firmer  Footing,  but 
Progress  Still  Lags 


Why  GAO  Did  This  Study 

The  F-35  Lightning  II,  also  known  as 
the  Joint  Strike  Fighter  (JSF),  is  the 
Department  of  Defense’s  (DOD)  most 
costly  and  ambitious  aircraft 
acquisition,  seeking  to 
simultaneously  develop  and  field 
three  aircraft  variants  for  the  Air 
Force,  Navy,  Marine  Corps,  and  eight 
international  partners.  The  JSF  is 
critical  for  recapitalizing  tactical  air 
forces  and  will  require  a  long-term 
commitment  to  very  large  annual 
fimding  outlays.  The  current 
estimated  investment  is  $382  billion 
to  develop  and  procure  2,457  aircraft. 

This  report,  prepared  in  response  to  a 
congressional  mandate  in  the 
National  Defense  Authorization  Act 
for  Fiscal  Year  2010,  discusses 

(1)  program  cost  and  schedule 
changes  and  their  implications  on 
affordability;  (2)  progress  made 
during  2010;  (3)  design  and 
manufacturing  maturity;  and  (4)  test 
plans  and  progress.  GAO’s  work 
included  analyses  of  a  wide  range  of 
program  documents  and  interviews 
with  defense  and  contractor  officials. 

What  GAO  Recommends 

To  sustain  a  focus  on  accountability 
and  facilitate  tradeoffs  within  the  JSF 
program,  GAO  recommends  that 
DOD  (1)  maintain  annual  funding 
levels  at  current  budgeted  amounts; 

(2)  establish  criteria  for  evaluating 
the  STOVL’s  progress  and  make 
independent  reviews,  allowing  each 
variant  to  proceed  at  its  own  pace; 
and  (3)  conduct  an  independent 
review  of  the  software  development 
and  lab  accreditation  processes.  DOD 
concurred. 

View  GAO-1 1-325  or  key  components. 

For  more  information,  contact  Michael  J. 
Sullivan  at  (202)  512-4841  or 
sullivanm@gao.gov 


What  GAO  Found 

DOD  continues  to  substantially  restructure  the  JSF  program,  taking  positive 
actions  that  should  lead  to  more  achievable  and  predictable  outcomes. 
Restructuring  has  consequences — higher  up-front  development  costs,  fewer 
aircraft  in  the  near  term,  training  delays,  and  extended  times  for  testing  and 
delivering  capabilities  to  warfighters.  Total  development  funding  is  now  $56.4 
billion  to  complete  in  2018,  a  26  percent  increase  in  cost  and  a  5-year  slip  in 
schedule  compared  to  the  current  baseline.  DOD  also  reduced  procurement 
quantities  by  246  aircraft  through  2016,  but  has  not  calculated  the  net  effects 
of  restructuring  on  total  procurement  costs  nor  approved  a  new  baseline. 
Mfordability  for  the  U.S.  and  partners  is  challenged  by  a  near  doubling  in 
average  imit  prices  since  program  start  and  higher  estimated  life-cycle  costs. 
Going  forward,  the  JSF  requires  unprecedented  funding  levels  in  a  period  of 
more  austere  defense  budgets. 

The  program  had  mixed  success  in  2010,  achieving  6  of  12  major  goals  it 
established  and  making  varying  degrees  of  progress  on  the  others.  Successes 
included  the  first  flight  of  the  carrier  variant,  award  of  a  fixed-price  aircraft 
procurement  contract,  and  an  accelerated  pace  in  development  flight  tests 
that  accomplished  three  times  as  many  flights  in  2010  as  the  previous  3  years 
combined.  However,  the  program  did  not  deliver  as  many  aircraft  to  test  and 
training  sites  as  planned  and  made  only  a  partial  release  of  software 
capabilities.  The  short  take  off  and  landing  variant  (STOVL)  experienced 
significant  technical  problems  and  did  not  meet  flight  test  expectations.  The 
Secretary  of  Defense  directed  a  2-year  period  to  evaluate  and  engineer  STOVL 
solutions. 

After  more  than  9  years  in  development  and  4  in  production,  the  JSF  program 
has  not  fully  demonstrated  that  the  aircraft  design  is  stable,  manufacturing 
processes  are  mature,  and  the  system  is  reliable.  Engineering  drawings  are 
still  being  released  to  the  manufacturing  floor  and  design  changes  continue  at 
higher  rates  than  desired.  More  changes  are  expected  as  testing  accelerates. 
Test  and  production  aircraft  cost  more  and  are  taking  longer  to  deliver  than 
expected.  Manufacturers  are  improving  operations  and  implemented  8  of  20 
recommendations  from  an  expert  panel,  but  have  not  yet  demonstrated  a 
capacity  to  efficiently  produce  at  higher  production  rates.  Substantial 
improvements  in  factory  throughput  and  the  global  supply  chain  are  needed. 

Development  testing  is  still  early  in  demonstrating  that  aircraft  will  work  as 
intended  and  meet  warfighter  requirements.  Only  about  4  percent  of  JSF 
capabilities  have  been  completely  verified  by  flight  tests,  lab  results,  or  both. 
Only  3  of  the  extensive  network  of  32  ground  test  labs  and  simulation  models 
are  fully  accredited  to  ensure  the  fidelity  of  results.  Software  development — 
essential  for  achieving  about  80  percent  of  the  JSF  functionality — is 
significantly  behind  schedule  as  it  enters  its  most  challenging  phase. 
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April  7,  2011 

Congressional  Committees 

The  F-35  Lightning  II,  also  known  as  the  Joint  Strike  Fighter  (JSF)  is  the 
Department  of  Defense’s  (DOD)  most  costly  and  ambitious  aircraft 
acquisition,  seeking  to  simultaneously  develop  and  field  three  aircraft 
variants  for  the  Air  Force,  Navy,  Marine  Corps,  and  eight  international 
partners.  The  JSF  is  the  core  of  DOD’s  long  term  tactical  aircraft 
recapitalization  plans  as  it  is  intended  to  replace  hundreds  of  legacy 
aircraft.  Total  U.S.  planned  investment  in  the  JSF  is  about  $385  billion  to 
develop  and  acquire  2,457  aircraft  through  2035.  With  that  many  dollars  at 
stake  amidst  pressing  warfighter  requirements  for  this  new  5th  generation 
capability,  the  Department  has  lately  recognized  numerous  technical, 
financial,  and  management  shortcomings  and  continues  to  significantly 
restructure  the  program,  adding  more  time  and  money  and  making  other 
changes  that  we  support. 

GAO  has  reported  on  JSF  issues  for  a  number  of  years.  Appendix  2 
summarizes  key  findings  and  recommendations  from  that  body  of  work. 
One  recurring  theme  has  been  the  “single  step,”  or  revolutionary, 
acquisition  strategy  the  JSF  program  adopted  to  develop  and  acquire  full 
combat  capabilities  on  a  very  aggressive,  risky  schedule  with  substantial 
concurrency,  or  overlap  among  development,  testing,  and  production 
activities. '  That  strategy,  coupled  with  a  management  environment  that 
was  slow  to  acknowledge  and  address  problems  with  needed  changes,  are 
prime  contributors  to  the  relatively  poor  cost,  schedule,  and  performance 
outcomes  experienced  to  date.  Our  March  2010  report^  discussed 
additional  cost  and  schedule  pressures,  unsatisfactory  performance  in 
manufacturing  and  delivering  aircraft,  and  concerns  about  not  meeting 
warfighter  requirements  on  time  and  in  quantity.  We  recommended  that 
(1)  DOD  complete  an  independent,  comprehensive  cost  and  schedule 
estimate  and  establish  it  as  the  official  program  of  record  for  plarming, 


^Rather  than  a  single  step  approach,  best  practices  and  current  DOD  acquisition  guidance 
recommend  that  complex  weapon  system  programs  instead  adopt  a  more  evolutionary 
acquisition  strategy,  developing  and  procuring  new  systems  incrementally  to  help  achieve 
better  program  outcomes  and  deliver  new  capabilities  to  the  warfighters  sooner. 

^GAO,  Joint  Strike  Fighter:  Additional  Costs  and  Delays  Risk  Not  Meeting  Warfighter 
Requirements  on  Time,  GAO-10-382  (Washington,  D.C.:  Mar.  19,  2010).  Refer  to  the  related 
products  section  for  a  list  of  prior  GAO  reports  and  testimonies. 
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budgeting,  and  congressional  reporting  purposes  and  (2)  reassess 
warfighter  requirements  to  determine  the  minimum  needs  (both 
capabilities  and  capacity)  that  can  be  achieved  within  realistic  and 
reasonable  timeframes  and,  if  necessary,  defer  some  capabilities  to  future 
increments.  In  addition,  we  suggested  that  Congress  consider  requiring 
DOD  to  establish  a  “system  maturity  matrix,”  a  management  tool  to  better 
measure  the  program’s  annual  progress  toward  key  objectives  to  improve 
oversight  and  better  inform  future  budget  decisions. 

This  is  our  second  report  required  by  law^  in  which  we  (1)  evaluate 
program  cost  and  schedule  changes  and  their  implications  on  affordability; 
(2)  identify  progress  made  in  2010  against  established  goals;  (3)  assess 
elements  of  design  stability  and  manufacturing  maturity  and  review 
production  results;  and  (4)  report  status  of  development  testing  and 
technical  challenges  facing  the  program.  To  conduct  this  work,  we 
evaluated  DOD’s  restructuring  actions  and  impacts  on  the  program, 
tracked  cost  and  schedule  changes,  and  determined  factors  driving  the 
changes.  We  reviewed  program  status  reports,  manufacturing  data,  test 
plans,  and  internal  DOD  analyses.  We  discussed  results  to  date  and  future 
plans  to  complete  JSF  development  and  move  further  into  procurement 
with  DOD,  JSF,  and  contractor  officials  including  members  of  the 
independent  review  teams.  We  toured  aircraft  and  engine  manufacturing 
plants,  obtained  production  and  supply  performance  indicators,  and 
discussed  improvements  underway  with  contractors.  We  conducted  this 
performance  audit  from  May  2010  to  March  2011  in  accordance  with 
generally  accepted  government  auditing  standards.  Those  standards 
require  that  we  plan  and  perform  the  audit  to  obtain  sufficient,  appropriate 
evidence  to  provide  a  reasonable  basis  for  our  findings  and  conclusions 
based  on  our  audit  objectives.  We  believe  that  the  evidence  obtained 
provides  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our 
audit  objectives. 


Background 


The  F-35  program  is  a  joint,  multinational  acquisition  to  develop  and  field 
an  affordable,  highly  common  family  of  stealthy,  next-generation  strike 
fighter  aircraft  for  the  United  States  Air  Force,  Marine  Corps,  Navy,  and 
eight  international  partners.  The  JSF  is  a  single-seat,  single-engine  aircraft 
incorporating  low-observable  (stealth)  technologies,  defensive  avionics. 


^National  Defense  Authorization  Act  for  Fiscal  Year  2010,  Pub.  L.  No.  111-84  §  244  (2009). 
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advanced  sensor  fusion/  internal  and  external  weapons,  and  advanced 
prognostic  maintenance  capability.  There  are  three  variants.  The  F-35A 
conventional  takeoff  and  landing  (CTOL)  variant  will  provide  air-to-ground 
attack  capabilities  to  replace  the  Air  Force’s  F-16  Fighting  Falcon  and  the 
A-10  Thunderbolt  11  aircraft,  and  will  complement  the  F-22A  Raptor.  The 
F-35B  short  takeoff  and  vertical  landing  (STOVL)  aircraft  will  be  a  multi¬ 
role  strike  fighter  to  replace  the  Marine  Corps’  F/A-18C/D  Hornet  and  AV- 
8B  Harrier  aircraft.  The  F-35C  carrier-suitable  variant  (CV)  will  provide  the 
Navy  and  Marine  Corps  a  multi-role,  stealthy  strike  aircraft  to  complement 
the  F/A-18E/F  Super  Hornet.  The  JSF  is  DOD’s  largest  cooperative 
program.^  Our  international  partners  are  providing  about  $5.1  billion 
toward  development,  and  foreign  firms  are  part  of  the  industrial  base 
producing  aircraft.  DOD’s  funding  requirements  for  the  JSF  assume 
economic  benefits  from  partner  purchases  in  reducing  unit  costs  for  U.S. 
aircraft. 

JSF  concept  development  began  in  November  1996  with  a  5-year 
competition  between  contractors  to  determine  the  most  capable  and 
affordable  preliminary  aircraft  design.  Lockheed  Martin  won  the 
competition  and  the  JSF  program  entered  system  development  and 
demonstration  in  October  2001.  Pratt  and  Whitney  is  the  primary  engine 
manufacturer,  while  General  Electric  has  been  developing  a  potential 
second  source  for  the  engine.  System  integration  efforts  and  a  preliminary 
design  review  then  revealed  significant  airframe  weight  problems 
impacting  key  performance  requirements.  In  March  2004,  DOD  rebaselined 
the  program,  adding  time  and  money  for  development  and  delaying  key 
milestones.  The  Navy  and  Marine  Corps  also  reduced  their  planned 
procurement  by  409  jets,  reducing  the  total  U.S.  buy  to  the  current  2,457 
quantity.  The  program  was  again  rebaselined  in  March  2007  to  reflect 
additional  cost  increases  and  schedule  slips  and  the  procurement  period 
was  extended  by  7  years  to  2034  with  reduction  in  aimual  quantities. 

Because  of  continuing  problems  and  poor  outcomes,  the  Secretary  of 
Defense  armounced  another  comprehensive  restructuring  of  the  JSF 
program  in  February  2010.  The  restructuring  followed  an  extensive 


''  Sensor  fusion  is  the  ability  to  take  information  from  both  multiple  onboard  and  off  board 
aircraft  sensors  and  display  the  information  in  an  easy-to-use  format  for  the  single  pilot. 

^  The  international  partners  are  the  United  Kingdom,  Italy,  the  Netherlands,  Turkey, 
Canada,  Australia,  Denmark,  and  Norway.  These  nations  are  contributing  funds  for  system 
development  and  plan  to  procure  more  than  700  aircraft. 
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Department-wide  review  initiated  in  2009  and  considered  the  findings  and 
recommendations  from  three  independent  groups  chartered  to  assess  the 
program:  the  Joint  Estimating  Team  (JET)  evaluated  program  execution 
and  resource  requirements;  the  Independent  Manufacturing  Review  Team 
(IMRT)  assessed  contractor  capabilities  and  plans  for  ramping-up  and 
sustaining  production  at  maximum  rates;  and  the  Joint  Assessment  Team 
(JAT)  reviewed  engine  costs  and  affordability  initiatives.  Key 
restructuring  actions  included  adding  $2.8  billion  for  development, 
extending  flight  testing  by  13  months,  adding  flight  test  resources  (one 
new  test  jet  and  use  of  3  production  jets),  reduced  near-term  procurement 
by  122  aircraft,  and  review  of  the  military  services’  capability  need  dates. 
The  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
stated  that  the  department-wide  review  would  continue  under  new 
program  management  and  cited  2010  as  a  critical  year  for  assessing 
progress  against  the  new  plans  and  the  expected  delivery  of  all  test 
aircraft,  completion  of  hundreds  of  test  flights,  and  meeting  other  key 
milestones.  We  supported  these  actions  in  our  March  2010  report  and 
subsequent  testimonies.®  We  noted  the  likelihood  of  additional  cost 
growth  and  schedule  extensions  as  the  restructuring  continues. 

In  March  2010,  the  Department  declared  that  the  program  experienced  a 
breach  of  the  critical  cost  growth  statutory  thresholds.^  The  Department 
subsequently  certified  to  Congress  in  June  2010  that  the  JSF  program 
should  continue.®  Table  1  summarizes  the  evolution  of  JSF  cost  and 


®  GAO-lO-382.  Also  GAO,  Joint  Strike  Fighter:  Significant  Challenges  and  Decision 
Ahead,  GAO-10-478T  (Washington,  D.C.:  Mar.  24,  2010)  and  GAO,  Joint  Strike  Fighter: 
Significant  Challenges  Ahead  as  DOD  Restructures  Program,  GAO-10-520T  (Washington, 
D.C.:  Mar.  11,  2010). 

’’  Commonly  referred  to  as  Nunn-McCurdy,  10  U.S.C.  §  2433  establishes  the  requirement  for 
DOD  to  submit  unit  cost  reports  on  major  defense  acquisition  programs  or  designated 
major  subprograms.  Two  measures  are  tracked  against  the  current  and  original  baseline 
estimates  for  a  program:  procurement  unit  cost  (total  procurement  funds  divided  by  the 
quantity  of  systems  procured)  and  program  acquisition  unit  cost  (total  funds  for 
development,  procurement,  and  system-specific  military  construction  divided  by  the 
quantity  of  systems  procured).  If  a  program’s  procurement  unit  cost  or  acquisition  unit  cost 
increases  by  at  least  25  percent  over  the  current  basehne  estimate  or  at  least  50  percent 
over  the  original  baseline  estimate,  it  constitutes  a  breach  of  the  critical  cost  growth 
threshold.  Programs  are  required  to  notify  Congress  if  a  Nunn-McCurdy  breach  is 
experienced. 

®  When  a  program  experiences  a  Nurm-McCurdy  breach  of  the  critical  cost  growth 
threshold,  DOD  is  required  to  take  a  number  of  steps  including  reeissessing  the  program 
and  submitting  a  certification  to  Congress  in  order  to  continue  the  program,  in  accordance 
with  10  U.S.C.  §  2433a. 
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schedule  estimates  at  key  junctures  in  its  acquisition  history  through  the 
current  Nunn-McCurdy  certification.  Since  then,  in  January  2011,  the 
Secretary  of  Defense  announced  additional  development  cost  increases 
and  further  changes  consequent  to  the  ongoing  restructure,  but  has  not  yet 
established  a  new  approved  acquisition  program  baseline. 


Table  1 :  Changes  in  Reported  JSF  Program  Cost,  Quantities,  and  Deliveries 

October 

December  March  2007 

April  2010 

2001  (system 

2003  (2004 

(approved 

(initial  program 

June  2010 

development  start) 

replan) 

baseline) 

restructure) 

(Nunn-McCurdy) 

Expected  quantities 

Development  quantities 

14 

14 

15 

14 

14 

Procurement  quantities 
(U.S.  only) 

2,852 

2,443 

2,443 

2,443 

2,443 

Total  quantities 

2,866 

2,457 

2,458 

2,457 

2,457 

Cost  estimates  (then-year  dollars  in  billions) 


Development 

$34.4 

$44.8 

$44.8 

$50.2 

$51.8 

Procurement 

196.6 

199.8 

231.7 

277.5 

325.1 

Military  construction 

2.0 

0.2 

2.0 

0.6 

5.6 

Total  program  acquisition 

$233.0 

$244.8 

$278.5 

$328.3 

$382.5 

Unit  cost  estimates  (then-year  dollars  in  millions) 


Program  acquisition 

$81 

$100 

$113 

$134 

$156 

Average  procurement 

69 

82 

95 

114 

133 

Estimated  delivery  and  production  dates 

First  operational  aircraft 
delivery 

2008 

2009 

2010 

2010 

2010 

Initial  operational  capability 

2010-2012 

2012-2013 

2012-2015 

2012-2016 

TBD 

Full-rate  production 

2012 

2013 

2013 

2016 

2016 

Source:  GAO  analysis  and  DOD  data. 

Note:  Does  not  reflect  cost  and  schedule  effects  from  additional  restructuring  actions  announced  after 
J  une  2010. 
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Affordability 
Expectations  Are 
Challenged  as  JSF 
Acquisition  Costs  Rise 
and  Schedules  Slip 


Ongoing  JSF  restructuring  continues  to  add  more  cost  and  time  for 
developing,  testing,  and  delivering  aircraft  to  the  warfighter.  These 
actions,  effectively  implemented,  should  result  in  more  predictable  and 
achievable  program  outcomes,  but  restructuring  comes  with 
consequences — higher  upfront  development  costs,  fewer  aircraft  received 
in  the  near  term,  training  delays,  and  extended  times  for  testing  and 
delivering  the  capabilities  required  by  the  warfighter.  Affordability  for  the 
U.S.  and  our  allies  is  challenged  because  unit  prices  are  about  double  what 
they  were  at  program  start  and  with  new  forecasts  that  the  aircraft  may 
cost  substantially  more  to  operate  and  maintain  over  the  life  cycle  than  the 
legacy  aircraft  they  replace.  Going  forward,  the  program  requires 
unprecedented  levels  of  funding  in  a  period  of  more  austere  defense 
funding.  Defense  leaders  stated  that  the  JSF  program  lost  its  focus  on 
affordability  and  that  restoring  the  focus  is  paramount  to  improving 
program  outcomes. 


Additional  Cost  Increases 
and  Schedule  Delays  Are 
Expected  as  Program 
Restructuring  Continues 


Defense  leadership  continued  to  restructure  the  JSF  program  following 
the  Nunn-McCurdy  certification.  In  January  2011,  the  Secretary  of  Defense 
directed  additional  changes,  stemming  in  large  part  from  the  results  of  a 
comprehensive  technical  baseline  review  under  new  government  and 
contractor  management.  Key  program  changes  (1)  added  $4.6  billion  to 
the  development  program  through  completion  for  a  total  development 
program  estimate  of  $56.4  billion  (an  increase  of  26  percent  against  the 
current  baseline  and  64  percent  from  the  original  baseline  at  program 
start);  (2)  extended  the  development  test  period  to  2016  (a  4-year  slip  from 
the  current  baseline);  and  (3)  reduced  near-term  procurement  quantities 
by  124  aircraft  in  addition  to  the  122  aircraft  cut  announced  in  February 
2010;  and  (4)  lowered  the  annual  rate  of  increase  for  boosting  future 
production. 


Because  of  the  lingering  technical  issues  on  the  STOVL,  the  most  complex 
variant,  the  Secretary  decoupled  STOVL  flight  tests  from  the  combined 
test  plan  and  scaled  back  STOVL  production  to  only  3  in  fiscal  year  2011 
and  to  6  per  year  for  fiscal  years  2012  and  2013.  This  represents  a  total  cut 
of  37  STOVL  aircraft  during  this  3-year  period  compared  to  the  fiscal  year 
2011  budget  plans.  In  announcing  these  changes,  the  Secretary  also  noted 
the  STOVL’s  significant  testing  problems  which  include  lift  fan  engine 
deficiencies,  and  poor  durability  test  results,  which  could  require 
redesigns  and  add  weight  to  aircraft’s  structure  and  propulsion  system. 
While  the  Secretary  decoupled  STOVL  from  the  flight  test  program,  STOVL 
was  not  further  separated  from  the  rest  of  the  JSF  program  for 
management  and  reporting  activities.  It  remains  a  part  of  the  combined 
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JSF  program  for  milestone  decisions  and  cost,  schedule,  and  performance 
reporting.  Resolving  STOVL  problems  and  moving  forward  at  an 
affordable  cost  is  essential  to  the  Marine  Corps’  future  plans,  which 
depend  upon  acquiring  the  STOVL  in  quantity  to  directly  accompany, 
protect,  and  provide  firepower  to  its  ground  expeditionary  forces. 

The  recently  submitted  fiscal  year  2012  Defense  Budget  reflects  the 
financial  impacts  from  restructuring  actions  through  2016.  Compared  to 
estimates  in  the  fiscal  year  2010  future  years  defense  program  for  the  same 
5-year  period,  the  Department  increased  development  funding  by 
$7.7  billion  and  decreased  procurement  funding  by  $8.4  billion  reflecting 
plans  to  buy  fewer  aircraft.  Table  2  summarizes  the  revised  development 
and  procurement  funding  requirements  and  annual  quantities  following 
the  Secretary’s  reductions.  Even  after  decreasing  near-term  quantities  and 
lowering  the  ramp  rate,  JSF  procurement  still  rapidly  increases.  Annual 
funding  levels  more  than  double  and  quantities  more  than  triple  during  this 
period.  These  numbers  do  not  include  the  additional  orders  expected  from 
the  international  partners. 


Table  2:  Budgeted  Development  and  Procurement  Funding  and  Quantities  for  Fiscal  Years  2012-2016 

(Dollars  In  billions) 

Development  Funding 

2012 

2013 

2014 

2015 

2016 

Total 

Air  Force  (CTOL) 

$1.4 

$1.2 

$0.9 

$0.6 

$0.4 

$4.5 

Navy  (CV) 

0.7 

0.7 

0.7 

0.6 

0.5 

3.2 

Marine  Corps  (STOVL) 

0.7 

0.7 

0.7 

0.6 

0.5 

3.2 

U.S.  total  development 

$2.7 

$2.7 

$2.3 

$1.8 

$1.3 

$10.8 

Procurement  Funding 

2012 

2013 

2014 

2015 

2016 

Total 

Air  Force  (CTOL) 

$3.8 

$4.1 

$5.6 

$6.5 

$8.5 

$28.5 

Navy  (CV) 

1.8 

2.5 

2.8 

3.3 

2.9 

13.2 

Marine  Corps  (STOVL) 

1.3 

1.3 

1.4 

2.0 

2.9 

9.0 

U.S.  total  procurement 

$6.9 

$7.9 

$9.8 

$11.8 

$14.3 

$50.7 

Procurement  Quantities 

2012 

2013 

2014 

2015 

2016 

Total 

Air  Force  (CTOL) 

19 

24 

40 

50 

70 

203 

Navy  (CV) 

7 

12 

14 

19 

20 

72 

Marine  Corps  (STOVL) 

6 

6 

8 

12 

18 

50 

U.S.  total  quantity 

32 

42 

62 

81 

108 

325 

Source:  GAO  analysis  of  fiscal  year  2012  President's  Budget 

Note:  Numbers  may  not  add  due  to  rounding. 
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Additional  changes  to  cost  and  schedule  are  likely  as  restructuring 
continues.  At  the  time  of  this  report,  the  Secretary  had  not  yet  granted  new 
milestone  B  approval  nor  approved  a  new  acquisition  program  baseline. 
Originally  planned  for  November  2010,  program  officials  now  expect  the 
new  acquisition  program  baseline  in  late  2011.  Also,  cost  analysts  are  still 
revising  procurement  funding  requirements  for  the  period  fiscal  year  2017 
through  completion  of  procurement  in  2035.  Accordingly,  the  net  effect  of 
reducing  near-term  procurement  quantities  and  deferring  these  aircraft  to 
future  years  is  uncertain  and  depends  upon  the  assumptions  made  about 
future  unit  prices,  annual  quantities,  and  inflation.  We  expect  total 
procurement  costs  will  be  somewhat  higher  than  the  estimate  submitted  in 
the  Nunn-McCurdy  certification  (refer  to  table  1).  Reduced  quantities  and 
use  of  production  aircraft  in  testing  will  also  limit  training  activities  for  the 
near-term  and  delay  deliveries  of  new  capabilities  to  the  warfighters. 
Officials  now  forecast  that  the  completion  of  system  development, 
completion  of  initial  operational  testing,  and  the  full  rate  production 
decision  will  extend  into  2018.  This  represents  slips  of  about  5  years  in 
these  important  milestones  against  the  current  program  baseline  approved 
in  2007. 

The  military  services  are  evaluating  the  impacts  from  restructuring  on 
their  initial  operational  capability  (IOC)  milestones,  the  critical  need  dates 
when  the  warfighter  must  have  in  place  the  first  increment  of  operational 
forces  available  for  combat.  In  response  to  the  initial  set  of  restructuring 
actions,  the  Air  Force  and  Navy  tentatively  extended  these  milestones  to 
2016,  but  the  Marine  Corps  slightly  adjusted  its  IOC  date  by  9  months  to 
December  2012.  It  is  all  but  certain  that  the  Marine  Corps  will  be  delaying 
its  IOC  date  in  the  wake  of  the  Secretary’s  STOVL  actions.  Air  Force  and 
Navy  dates  may  also  be  adjusted  to  reflect  the  newest  developments. 


Program  Requires 
Unprecedented  Funding 
Levels  Well  into  the  Future 


Affordability — both  in  terms  of  the  investment  costs  to  acquire  the  JSF 
and  the  continuing  costs  to  operate  and  maintain  it  over  the  life-cycle — is 
at  risk.  A  key  tenet  of  the  JSF  program  from  its  inception  has  been  to 
deliver  an  affordable,  highly  common  fifth  generation’’  aircraft  that  could 
be  acquired  by  the  warfighters  in  large  numbers.  Rising  aircraft  prices 
erode  buying  power  and  make  it  difficult  for  the  U.S.  and  its  allies  to  buy 
as  many  aircraft  as  planned.  Quantity  reductions  could  drive  additional 


^  Rfth  generation  aircraft  include  the  F-22A  and  JSF  and  incorporate  stealth 
characteristics,  fused  sensor  data,  and  advanced  radars. 
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price  increases  for  future  aircraft.  Further,  while  the  Department  is  still 
refining  cost  projections  for  operating  and  supporting  future  JSF  fleets, 
cost  forecasts  have  increased  as  the  program  matures  and  more  data 
becomes  available.  Current  JSF  life-cycle  cost  estimates  are  considerably 
higher  than  the  legacy  aircraft  it  will  replace;  this  has  major  implications 
for  future  demands  on  military  operating  and  support  budgets  and  plans 
for  recapitalizing  fighter  forces.  Defense  leadership  stated  that  the  JSF 
program  lost  focus  on  affordability  and  that  restoring  and  maintaining  that 
focus  is  paramount  to  improving  program  outcomes. 

In  light  of  continued  cost  growth,  the  program  places  unprecedented 
demands  for  funding  in  the  defense  budget — an  annual  average  of  almost 
$11  billion  for  the  next  two  decades.  (This  and  other  data  in  this  paragraph 
reflect  the  fiscal  year  2011  budget  submission.)  During  the  peak  years  of 
production,  the  average  annual  requirement  is  about  $13  billion.  The  JSF 
will  have  to  annually  compete  with  other  defense  and  nondefense 
priorities  for  the  shrinking  discretionary  federal  dollar  amid  continued 
concerns  about  the  national  debt  and  long  term  fiscal  pressures.  The  JSF 
program  has  received  more  than  $56  billion  through  fiscal  year  2010.  To 
complete  the  acquisition  program  as  currently  planned,  another 
$272  billion  will  be  required  from  2011  through  2035.  Figure  1  illustrates 
the  annual  funding  requirements  outlined  in  the  program’s  Selected 
Acquisition  Report  released  in  April  2010.  These  funding  levels  do  not 
reflect  the  additional  funding  increases  in  the  Nunn-McCurdy  certification 
and  the  Secretary’s  recent  actions.  DOD  is  in  the  process  of  establishing  a 
new  acquisition  program  baseline  which  will  likely  project  even  higher 
funding  requirements. 
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Figure  1 :  JSF  Annual  Development  and  Procurement  Funding  Requirements  (April  2010  Estimate) 


Funding  requirements  (doiiars  in  biiiions) 
$16 


Fiscai  year 


Source:  GAO  analysis  of  DOD  data. 


The  JSF  is  the  hnchpin  in  DOD’s  tactical  aircraft  recapitalization  plans, 
replacing  hundreds  of  legacy  aircraft.  Because  of  its  sheer  size  and  high 
priority  within  the  Department,  even  relatively  modest  cost  growth  on  the 
JSF  can  require  the  sourcing  of  billions  of  additional  funds,  largely  from 
other  programs  in  DOD’s  acquisition  portfolio.  On  the  other  hand,  slips  in 
JSF  schedules,  cuts  in  armual  procurement  quantities,  and  deferred 
delivery  of  operational  aircraft  can  require  additional  monies  be  spent  on 
legacy  aircraft,  postponing  planned  retirements  and  sustaining  fleets  for 
longer  periods  of  time.  To  mitigate  projected  shortfalls  in  tactical  aircraft 
inventories  due  to  JSF  perturbations,  the  Navy  recently  procured 
additional  F/A-18E/F  Super  Hornets  and  both  the  Navy  and  Air  Force  are 
funding  service  life  extension  programs  and  adding  new  capabilities  to 
legacy  aircraft. 

Furthermore,  international  partners’  participation  in  the  JSF  program  is 
very  important  to  maintaining  affordability  for  all  buyers.  DOD  budget 
plans  expect  the  partners  to  buy  223  aircraft  costing  $24.1  billion  during 
the  fiscal  year  2011-2016  period.  However,  JSF  cost  increases,  schedule 
delays,  and  internal  issues  may  result  in  reduced  or  deferred  foreign  buys. 
Some  partners  have  already  signaled  plans  to  buy  fewer  aircraft,  a 
different  mix  of  aircraft,  or  defer  purchases  to  later  years.  On  the  positive 
side,  other  countries  have  expressed  interest  in  acquiring  the  JSF. 
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Decisions  made  by  the  international  community  and  its  impact  on  JSF 
affordability  are  largely  beyond  the  program’s  direct  control.  However, 
improving  JSF  program  outcomes  to  lower  costs  and  reassure  buyers  is 
within  DOD’s  and  the  contractors’  control. 

The  eight  international  partners  have  important  stakes  in  the  JSF  program, 
having  provided  about  $5  billion  in  development  funding,  being  counted 
upon  to  procure  hundreds  of  aircraft,  and  expecting  their  industries  to 
receive  a  significant  portion  of  JSF  manufacturing  and  supply  business. 
DOD’s  procurement  cost  estimates  provided  to  the  Congress  have  long 
assumed  that  the  eight  partners  will  buy  at  least  730  JSF  aircraft.  Unit 
prices  for  U.S.  quantities  assume  the  economic  benefit  of  these  purchases. 
If  fewer  are  sold  overseas,  the  Air  Force,  Navy  and  Marine  Corps  (and  the 
American  taxpayer)  may  have  to  pay  more.  Unit  costs  can  be  expected  to 
increase  with  smaller  purchases  due  to  diminished  manufacturing 
economies  of  scale  and  because  fixed  costs  have  to  be  spread  over  fewer 
aircraft. 

Maintaining  a  strong  focus  on  affordability  necessitates  having  reliable  and 
complete  cost  data  that  provides  accurate  accounting  reports,  identifies 
potential  cost  and  schedule  problems  early,  and  produces  sound  estimates 
of  the  cost  to  complete  work.  The  JSF  program  has  been  hampered  in  this 
regard  because,  for  at  least  the  past  three  years,  the  prime  contractor  has 
not  had  an  adequate  and  disciplined  earned  value  management  (EVM) 
system  in  place  to  effectively  track  costs  and  control  schedule.  The  prime 
contractor  was  found  deficient  in  meeting  19  of  32  required  guidelines, 
calling  into  question  its  ability  to  manage  the  escalating  costs  and  complex 
scheduling  of  the  JSF  program.  In  October  2010,  the  Defense  Contract 
Management  Agency  (DCMA)  withdrew  the  determination  of  compliance 
for  the  prime  contractor’s  EVM  system  due  to  longstanding  non- 
compliance  issues  with  specific  guidelines  that  underpin  a  sound  system.  “ 
To  address  these  shortcomings,  the  contractor  is  developing  new 
processes,  tools,  training,  and  enforcement  in  order  to  achieve  a  fully 
integrated  and  automated  EVM  system.  Officials  will  reassess  the  earned 


American  National  Standards  Institute/Electronics  Industries  Alliance-748  is  a  collection 
of  32  earned  value  management  system  guidelines  that  incorporate  business  best  practices 
for  program  management  systems  proven  to  provide  strong  benefits  for  program  or 
enterprise  planning  and  control.  The  processes  include  integration  of  program  scope, 
schedule,  and  cost  objectives,  establishment  of  a  baseline  plan  for  accomplishment  of 
program  objectives,  and  use  of  earned  value  techniques  for  performance  measurement 
during  the  execution  of  a  program.  The  system  provides  a  sound  basis  for  problem 
identification,  corrective  actions,  and  management  replanning  as  required. 
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value  system  by  March  2012 — more  than  four  years  after  these  problems 
were  first  discovered  to  see  if  modifications  needed  have  been  made. 

EVM  is  an  important,  established  tool  that  can  provide  objective  product 
status  reports.  DOD  requires  its  use  by  major  defense  suppliers  to 
facilitate  good  insight  and  oversight  of  the  expenditure  of  government 
dollars,  thereby  improving  both  affordability  and  accountability.  JSF  is 
DOD’s  largest  acquisition  ever,  so  it  is  particularly  critical  to  improve  and 
certify  the  contractor’s  EVM  system  as  expeditiously  as  possible.  If  not 
improved,  inaccurate  performance  reports  and  late  notice  of  cost  overruns 
will  likely  continue  to  hinder  timely  decision  making  and  corrective 
actions.  Strong  leadership  and  a  shared  vision  among  stakeholders  are 
critical  to  implementing  EVM  effectively. 


Progress  in  Achieving 
the  JSF  Program’s 
2010  Goals  Was  Mixed 


The  JSF  program  established  12  clearly-stated  goals  in  testing,  contracting, 
and  manufacturing  for  completion  in  calendar  year  2010.  It  had  mixed 
success,  achieving  6  goals  and  making  varying  degrees  of  progress  on  the 
other  6.  For  example,  the  program  exceeded  its  goal  for  the  number  of 
development  flight  tests  but  did  not  deliver  as  many  test  and  production 
aircraft  as  planned.  Also,  the  program  awarded  its  first  fixed-price 
contract  on  its  fourth  lot  of  aircraft  production,  but  did  not  award  the 
fixed-price  engine  contract  in  2010  as  planned.  Table  3  summarizes  JSF 
goals  and  accomplishments  for  2010. 
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Table  3:  JSF  Progress  on  Stated  Goals  for  2010 

Key  event 

Achieved 
in  2010 

Status 

Complete  400  development  flight  tests 

Yes 

Completed  410  test  flights 

First  vertical  landing  of  STOVL  variant 

Yes 

Achieved  March  2010 

Carrier  variant  first  flight 

Yes 

Achieved  J  une  2010 

Autonomic  logistic  information  system  is  operational 

Yes 

Began  limited  operations  J  uly  2010 

Training  for  125  maintenance  personnel  completed 

Yes 

Trained  138  maintenance  personnel 

Award  contract  for  fourth  aircraft  production  lot 

Yes 

Awarded  contract  November  2010 

Eleven  test  aircraft  delivered  to  test  sites 

No 

Delivered  eight  aircraft 

Flight  test  rate  of  12  flights  per  aircraft  per  month 
demonstrated 

No 

Achieved  flight  test  rate  of  2  to  8  per  month 

At  least  3  aircraft  delivered  to  Eglin  Air  Force  Base 

No 

None  delivered,  expected  mid-2011 

Begin  flight  training  operations  at  Eglin  Air  Force  Base 

No 

Expected  September  2011 

Block  1.0  software  delivered  to  flight  test 

No 

Delivered  limited  capability  November  2010  with  full 
capability  expected  J  une  2011 

Award  contract  for  fourth  engine  production  lot 

No 

Expected  May  2011 

Source:  GAO  analysis  of  DOD  data. 


The  development  flight  test  program  significantly  ramped  up  operations  in 
2010,  accomplishing  three  times  as  many  test  flights  as  the  previous 
3  years  combined.  Table  4  summarizes  actual  flights,  hours,  and  test 
points"  flown  by  each  variant  compared  to  the  2010  plan. 


"Flight  test  points  are  specific,  quantifiable  objectives  in  flight  plans  that  are  needed  to 
verify  aircraft  design  £ind  performance. 
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Table  4:  Flight  Test  Performance  in  2010 

Conventional  takeoff 
and  landing  variant 

Short  takeoff  and 
vertical  landing  variant 

Carrier  variant 

Total 

Flight  tests 

Actual 

171 

212 

27 

410 

Planned 

112 

251 

31 

394 

Difference 

59 

(39) 

(4) 

16 

Flight  test  hours 

Actual 

290 

286 

41 

617 

Planned 

202 

409 

56 

667 

Difference 

88 

(123) 

(15) 

(50) 

Flight  test  points  flown 

Actual 

1373 

1924 

496 

3793 

Planned 

1064 

2438 

270 

3772 

Difference 

309 

(514) 

226 

21 

Source:  GAO  analysis  of  DOD  data. 


Although  still  hampered  as  in  prior  years  by  the  late  delivery  of  test 
aircraft,  flight  tests  substantially  increased  in  volume  and  pace  at  the  two 
main  government  test  sites — Edwards  Air  Force  Base,  California,  for 
CTOL  tests  and  Patuxent  River  Naval  Air  Station  for  STOVL  and  CV 
testing.  The  CTOL  variant  significantly  exceeded  plans  while  initial  testing 
of  the  carrier  variant  was  judged  satisfactory,  below  plans  for  the  number 
and  hours  of  flight  but  ahead  on  test  points  flown.  The  STOVL,  however, 
substantially  under-performed  in  flight  tests  and  experienced  significant 
technical  issues  unique  to  this  variant  that  could  add  to  its  weight  and 
cost.  The  STOVL’s  test  problems  were  a  major  factor  in  the  heightened 
scrutiny  and  two-year  probation  period  directed  by  the  Secretary  to 
engineer  solutions,  assess  impacts,  and  inform  a  future  decision  as  to 
whether  and  how  to  proceed  with  this  variant. 

Evaluating  armual  performance  against  stated  goals  can  be  an  effective 
tool  that  facilitates  oversight  by  the  Congress  and  defense  leadership  and 
useful  for  informing  future  budget  decisions.  In  our  2010  report,  we 
suggested  that  Congress  consider  requiring  DOD  to  establish  a  system 
maturity  matrix  to  better  measure  the  program’s  progress  in  maturing  the 
weapon  system  and  providing  evidence  to  support  budget  decisions.  The 


GAO-lO-382. 
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Ike  Skelton  National  Defense  Authorization  Act  for  Fiscal  Year  2011“ 
established  this  requirement  and  we  understand  the  Department  is 
working  on  its  implementation.  We  believe  this  tool  and  process  will 
improve  oversight  and  budgeting,  holding  people  accountable  for  meeting 
interim  objectives  and,  for  objectives  not  met,  providing  criteria  and  a 
forum  for  evaluating  reasons  why  and  what  should  be  done. 


Program  Has  Still  Not 
Fully  Demonstrated  a 
Stable  Design  and 
Mature  Manufacturing 
Processes  as  It  Enters 
Its  Fifth  Year  of 
Production 


After  completing  9  years  of  system  development  and  4  years  of 
overlapping  production  activities,  the  JSF  program  has  been  slow  to  gain 
adequate  knowledge  that  its  design  and  manufacturing  process  are  fully 
mature  and  ready  for  greater  levels  of  annual  production.  The  JSF 
program  still  lags  in  achieving  critical  indicators  of  success  expected  from 
well-performing  acquisition  programs.  Specifically,  the  program  has  not 
yet  stabilized  aircraft  designs — engineering  changes  continue  at  higher 
than  expected  rates  long  after  critical  design  reviews  and  well  into 
procurement,  and  more  changes  are  expected  as  testing  accelerates.  Also, 
the  aircraft  and  engine  manufacturing  processes  are  not  yet  mature 
enough  to  support  efficient  production  at  higher  annual  rates  and 
substantial  improvements  in  the  global  supply  network  are  needed. 
Further,  the  growth  in  aircraft  reliability — crucial  for  managing  life-cycle 
costs — has  not  been  demonstrated  to  the  extent  planned  by  this  time. 


The  Program  Has  Not  Yet  Engineering  drawings  released  since  design  reviews  and  the  number  and 

Stabilized  Aircraft  Designs  of  design  changes  are  excessive  compared  to  plans  and  best  practices. 

Critical  design  reviews  were  completed  on  the  three  aircraft  variants  in 
2006  and  2007  and  the  designs  declared  mature,  but  the  program  continues 
to  experience  numerous  changes.  Since  2007,  the  program  has  produced 
20,000  more  engineering  drawings,  a  50-percent  increase  in  total  drawings 
and  about  5  times  more  than  best  practices  suggest.  In  addition,  changes 
to  drawings  have  not  decreased  and  leveled  off  as  planned.  Figure  2 
tracks  and  compares  monthly  design  changes  and  future  forecasts  against 
contractor  plans  in  2007.  The  monthly  rate  in  2009  and  2010  was  higher 
than  expected  and  the  program  now  anticipates  more  changes  over  a 
longer  period  of  time — about  10,000  more  changes  through  January  2016. 
We  expect  this  number  to  go  up  given  new  forecasts  for  additional  testing 
and  extension  of  system  development  until  2018. 


“Pub.  L.  No.  111-383,  §  122. 
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Figure  2:  Monthly  Design  Changes  for  JSF  Aircraft 


Number  of  design  changes 
1000 

900 


Anticipated  design  changes  (2007  pian) 
Design  changes  (actuai) 

.  Anticipated  future  design  changes  (2010  pian) 

Source:  GAO  analysis  ofDOD  data. 


A  key  indicator  of  a  product’s  maturity  is  the  stability  of  its  design.  The 
number  of  engineering  drawings  released  and  subsequent  changes  provide 
indicators  of  the  maturity  of  the  design.  Engineering  drawings  are  critical 
because  they  communicate  to  the  manufacturer  and  suppliers  how  the 
part  functions,  what  it  looks  like,  and  what  materials  and  critical 
processes  are  used  to  build  the  product.  Best  practices  suggest  90  percent 
of  a  product’s  engineering  drawings  be  released  by  the  critical  design 
review.  Late  engineering  drawings  and  high  levels  of  changes  often 
indicate  a  lack  of  imderstanding  about  the  design,  and  can  cause  part 
shortages  and  inefficient  manufacturing  processes  as  work  is  performed 
out  of  sequence.  Some  level  of  design  change  is  expected  during  the 
production  cycle  of  any  new  and  highly  technical  product,  but  excessive 
changes  raise  questions  about  the  JSF’s  design  maturity  and  its  readiness 
for  higher  rates  of  production. 

With  most  of  development  testing  still  ahead  for  the  JSF,  the  risk  and 
impact  from  required  design  changes  are  significant.  Acquisition  programs 
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typically  encounter  higher  and  more  substantive  changes  as  a  result  of 
discovery  and  rework  during  development  flight  and  ground  testing. 
Future  changes  may  require  alterations  to  the  manufacturing  process, 
changes  to  the  supply  base,  and  costly  retrofitting  of  aircraft  already 
produced  and  fielded.  A  key  cost  driver  for  the  program  has  been  the 
higher  than  expected  effort  needed  to  address  design  related  issues.  The 
contractor  has  not  been  able  to  reduce  engineering  staff  as  fast  as 
expected.  DOD’s  restructuring  actions  recognize  these  issues  and  added 
time  to  development,  more  flight  testing,  and  reduced  procurement. 
Additional  changes  are  likely  as  development  flight  testing  continues. 

Some  emerging  concerns  may  drive  additional  and  substantive  design 
changes: 

•  JSF  Lift  System  Development  and  Integration.  Essential  to 
STOVL  operations,  the  lift  fan  continues  to  be  a  prime  risk  area.  The 
program  is  working  to  mature  lift  fan  and  drive  shaft  technologies  and 
a  required  redesign  expected  in  spring  2011. 

•  Fatigne  Cracks  in  STOVL  Test  Article.  During  a  recent  durability 
ground  test,  fatigue  cracks  were  discovered  in  a  major  bulkhead  of  the 
STOVL  test  article.  Cracks  were  discovered  after  1,500  hours  of 
durability  testing,  less  than  one-tenth  of  the  hours  planned  for  fatigue 
tests  to  certify  that  the  STOVL  airframe  meets  its  design  life 
requirement.  Officials  reported  that  stress  data  had  been  under¬ 
estimated  during  initial  design.  Inspections  of  aircraft  and  other  test 
articles  did  not  identify  cracks  at  the  same  site.  Decisions  about 
potential  redesign  and  re-manufacture  are  still  to  be  determined. 

•  Wing  Tip  Vortex.  Prime  contractor  officials  identified  wing  tip 
vortices  as  a  potential  risk  to  the  program.  Wing  tip  vortices  are  tubes 
of  circulating  air  which  are  left  behind  the  aircraft’s  wing  as  it 
generates  lift.  The  cores  of  the  vortices  are  sometimes  visible  because 
of  water  condensation.  If  these  are  visible  during  daytime  flights  they 
could  negatively  impact  the  aircraft’s  stealth  capabilities. 

•  Outer  Mold  Lines.  Defense  Contract  Management  Agency  officials 
noted  difficulties  in  manufacturing  outer  mold  lines,  resulting  from 
tight  tolerance  specifications  and  multiple  manufacturing 
methodologies  among  the  different  JSF  parts  suppliers.  The 
manufacturing  processes  are  new  and  different  from  legacy  practices. 
Inability  to  meet  the  outer  mold  line  requirements  could  have  major 
impacts  on  cost  as  well  as  stealth  requirements  and  capabilities.  This 
problem  is  not  expected  to  be  resolved  until  the  June  2015  time  frame 
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after  which  a  large  number  of  aircraft  will  have  been  built  and  would 
need  to  be  retrofitted  for  any  design  changes.  Program  officials  stated 
some  redesign  activities  have  begun  and  will  take  into  the  2013 
timeframe  to  begin  developing  the  changes,  their  costs,  and 
implementation.  The  effects  of  these  changes  could  extend  out  into 
2015,  but  will  be  prioritized  to  reduce  performance  and  cost  impacts. 


Manufacturing  Processes 
are  Not  Yet  Mature  Enough 
for  Efficient  Production  at 
Increased  Rates 


Manufacturing  and  delivering  test  jets  took  much  more  time  and  money 
than  planned  and  the  full  contingent  of  test  aircraft  is  still  not  available  at 
military  testing  sites,  years  later  than  promised.  Projected  costs  to 
complete  the  first  three  production  lots  for  aircraft  and  engines  also 
exceed  the  negotiated  amounts  at  contract  award  and  aircraft  will  be 
delivered  late.  The  production  impacts  of  restructuring  actions  that 
reduced  quantities,  lowered  the  ramp  rate,  and  delayed  the  full-rate 
production  decision  have  not  been  fully  determined.  We  found  that  the 
aircraft  and  engine  manufacturers  are  making  good  faith  efforts  to 
implement  the  IMRT  and  JAT  recommendations  and  to  make  other 
improvements  with  performance  measures  indicating  some  success. 


As  in  prior  years,  lingering  management  inefficiencies,  including 
substantial  out-of-station  work''*  and  part  shortages,  continued  to  increase 
the  labor  needed  to  manufacture  test  aircraft.  Figure  3  depicts  forecasted 
and  actual  labor  hour  requirements  for  building  12  production- 
representative  test  jets.  Total  labor  hours  required  to  produce  the  test 
aircraft  increased  over  time.  The  2010  actual  labor  hours  exceeded  the 
2007  budgeted  hours  by  more  than  1.5  million  hours,  a  75  percent  increase. 


Out  of  station  work  occurs  when  manufacturing  steps  are  not  completed  at  its  designated 
work  station  and  must  be  finished  elsewhere  later  in  production.  This  is  highly  inefficient, 
increasing  labor  hours,  causing  delays,  and  sometimes  quality  problems. 
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Figure  3:  JSF  Labor  Hours  for  Manufacturing  Test  Aircraft 
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Source:  GAO  analysis  of  DOD  data. 


Manufacturing  production  aircraft  is  different  from  building  test  aircraft, 
and  some  gains  in  learning  as  more  aircraft  are  built  can  be  expected  to, 
over  time,  reduce  labor  hour  costs.  However,  the  experience  to  date  on  the 
test  aircraft  and  initial  production  aircraft  suggests  that  future  costs  for 
building  production  aircraft  may  be  higher  than  currently  budgeted.  The 
costs  on  the  first  three  low-rate  production  contracts  have  increased  from 
amounts  negotiated  at  contract  award  and  the  completion  dates  for 
delivering  aircraft  have  been  extended  over  nine  months  on  average.  We 
are  encouraged  by  DOD’s  award  of  a  fixed  price  incentive  fee  contract  for 
lot  4  production  and  the  prospects  for  the  cost  study  to  inform  lot  5 
negotiations,  but  we  have  not  examined  contract  specifications. 

DOD  began  procuring  production  jets  in  2007  and  has  now  ordered  58 
aircraft  on  the  first  four  low-rate  initial  production  lots.  JSF  contracts 
anticipated  the  dehvery  of  14  production  jets  through  2010,  but  none  have 
been  delivered.  Delivery  of  the  first  two  production  jets  (both  CTOLs)  has 
been  delayed  several  times  since  the  contract  was  signed  and  is  now 
expected  in  April  2011.  In  addition,  DOD  expects  to  procure  32  more 
aircraft  in  fiscal  year  2011.  Building  a  large  backlog  of  jets  on  order  but 
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undelivered  is  not  an  efficient  use  of  federal  funds,  tying  up  millions  of 
dollars  in  obligations  ahead  of  the  ability  of  the  manufacturing  process  to 
produce.  We  note  that  the  Secretary  used  a  similar  line  of  reasoning  to 
reduce  STOVL  production. 

DOD  does  not  yet  know  the  full  effect  that  restructuring  actions  will  have 
on  future  armual  procurement  funding  requirements.  Cost  analysts  are 
still  calculating  the  impacts  from  deferring  procurement  of  246  aircraft 
from  the  near-term  to  future  years,  lowering  the  ramp  rate,  and  extending 
the  full-rate  production  decision.  Future  funding  requirements  could  be 
even  higher  than  projected  and  the  quantities  considered  affordable  by  the 
U.S.  and  allies  could  be  reduced,  further  driving  up  unit  costs. 

The  Secretary’s  decisions  to  reduce  near-term  procurement  quantities  and 
adopt  a  less-steep  ramp  up  in  future  production  were  based  on  IMRT 
findings.  The  Secretary  chartered  the  IMRT  to  comprehensively  review 
JSF  manufacturing  capacity  to  assess  the  contractor’s  ability  to  achieve 
plarmed  production  ramp-up  and  to  sustain  the  predicted  maximum 
production  rates.  The  IMRT’s  October  2009  report  made  20  specific 
recommendations  for  corrective  actions.  As  of  September  2010,  officials 
considered  eight  of  the  recommendations  complete  and  three  others  on 
track.  Implementation  of  the  remaining  nine  recommendations  was 
incomplete  or  behind  schedule. 

The  most  significant  incomplete  recommendation  is  improving  global 
supply  chain  management.  The  JSF  already  has  an  extensive  number  of 
suppliers  worldwide  and  those  numbers  will  increase  with  future 
workload  shared  among  numerous  domestic  and  foreign  firms.  The  IMRT 
cites  the  global  supply  chain  as  the  critical  manufacturing  challenge  facing 
the  program,  requiring  significant  improvement  in  delivery  performance 
and  responsiveness  in  order  to  achieve  the  program’s  eventual  production 
rate  goal  of  20  aircraft  per  month.  According  to  the  prime  contractor,  the 
global  supply  chain  remains  on  the  critical  path  and  progress  has  been 
made,  but  the  global  transportation  plan  and  supply  chain  risk 
management  plan  are  incomplete.  Another  IMRT  recommendation  that 
still  needs  to  be  addressed  is  the  performance  of  a  comprehensive 
schedule  risk  assessment,  now  expected  to  begin  in  spring  2011.  We 
recommended  this  in  our  March  2009  report.  “  Schedule  risk  assessments 


GAO,  Joint  Strike  Fighter:  Accelerating  Procurement  before  Completing  Development 
Increases  the  Government's  Financial  Risk,  GAO-09-303  (Washington  D.C.:  Mar.  12,  2009). 
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can  provide  keen  insight  into  critical  path  activities,  cost  and  schedule 
interrelationships,  and  emerging  risks. 

The  primary  F135  engine  contractor  faces  similar  challenges  as  it  moves 
deeper  into  production.  All  development  engines  and  initial  production 
units  have  been  delivered,  but  the  costs  to  complete  each  of  the  first  three 
engine  production  contracts  increased  and  deliveries  slipped  since 
contract  awards.  Officials  said  these  delays  have  not  been  especially 
troublesome  to  date  because  aircraft  deliveries  were  even  later.  The 
contractor  achieved  the  initial  service  release  for  the  CTOL  and  CV  engine, 
meaning  the  engine  configuration  is  qualified  and  ready  to  go  into 
production,  but  the  STOVL’s  initial  release  was  delayed  until  December 
2010  due  to  qualification  testing.  The  JAT  reviewed  F135  program 
performance,  identified  cost  drivers,  and  made  affordability  projections. 
JAT  officials  said  the  contractor’s  cost  reduction  efforts  were  credible  but 
largely  dependent  on  receiving  more  government  funding  for  affordability 
initiatives  and  alternative  sourcing  arrangements. 

Our  past  work  in  best  practices  found  that  successful  product 
development  programs  reach  a  point  at  which  they  know  that 
manufacturing  processes  will  efficiently  produce  a  new  product 
conforming  to  cost,  quality,  and  schedule  targets  before  they  begin 
producing  a  system.  Reaching  this  point  means  more  than  knowing  that 
the  product  can  be  built;  it  means  that  critical  manufacturing  processes 
are  under  control,  such  that  the  quality,  volume,  and  cost  are  proven 
acceptable.  By  these  criteria,  the  JSF  contractors’  abilities  to  ramp-up  to 
greater  rates  of  production  have  not  yet  been  demonstrated.  The  aircraft 
and  engine  manufacturers  now  have  significantly  more  items  in 
production  flow  compared  to  prior  years,  but  throughput  capacity  to 
complete  all  work  and  deliver  end  items  is  constrained.  We  determined 
that  the  aircraft  and  engine  contractors  are  making  good  faith  efforts  to 
implement  the  recommendations  of  the  IMRT  and  JAT  and  to  make  other 
improvements  to  production  capacity  and  flow.  The  aircraft  manufacturer 
is  reporting  a  decrease  in  out  of  station  work,  more  efficient  work  stations, 
improved  quality,  increased  parts  availability,  and  reduced  span  times. 
Until  improvements  are  fully  implemented  and  demonstrated,  the 
restructuring  actions  to  reduce  near  term  procurement  quantities  and 
establish  a  more  achievable  ramp  rate  was  appropriate  and  will  provide 
more  time  to  fully  mature  manufacturing  and  supply  processes  and  catch 
up  with  aircraft  backlogs.  Improving  factory  throughput  and  controlling 
costs — driving  down  unit  costs  and  delivering  on  time —  are  essential  for 
efficient  manufacturing  and  timely  delivery  to  the  warfighter  at  the 
increased  production  rates  planned  for  the  future. 
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Aircraft  Are  Not  Meeting 
Early  Reliability  Growth 
Plans 


STOVL  and  CTOL  aircraft  are  behind  reliability  growth  plans  aimed  at 
demonstrating  that  the  aircraft  will  meet  warfighter  support  and 
availability  requirements.  The  carrier  variant  is  in  early  stages  of  flight 
testing  and  sufficient  reliability  data  was  not  available.  Reliability  is  a 
function  of  the  specific  elements  of  a  product’s  design;  a  system  is  reliable 
when  it  can  perform  over  a  specified  period  of  time  without  failure, 
degradation,  or  need  of  repair.  Improvements  over  time  occur  through 
design  changes  or  manufacturing  process  improvements.  A  key  reliability 
metric  is  mean  flying  hours  between  failure,  defined  as  the  number  of 
flying  hours  achieved  divided  by  the  number  of  failures  incurred. 

Reliability  growth  plans  called  for  the  STOVL  to  have  achieved  at  least 
1.9  flying  hours  between  failures  and  for  the  CTOL  2.9  flying  hours 
between  failures  by  this  point  in  the  test  program.  However,  the  STOVL 
aircraft  is  significantly  behind  plans,  achieving  about  0.4  hours  between 
failures,  or  about  20  percent  of  what  was  expected  by  this  time.  The  CTOL 
variant  was  also  behind  plans  achieving  1.8  hours  between  failures, 
approximately  60  percent  of  what  was  expected.  Figure  4  depicts  progress 
of  each  variant  in  demonstrating  mean  flying  hours  between  failures,  as  of 
September  2010. 
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Figure  4:  JSF  Mean  Times  between  Failure  Demonstrated  to  Date 
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Source:  GAO  analysis  of  DoD  data. 


Improving  reliability  rates  are  essential  to  control  future  operating  costs 
and  ensine  aircraft  are  available  as  needed  by  the  warfighter.  Compared  to 
the  up-front  costs  of  acquiring  aircraft,  the  long-term  costs  for  operating, 
maintaining,  and  sustaining  JSF  fleets  over  an  aircraft’s  useful  life 
represent  the  much  larger  portion  of  total  ownership  costs.  We  have 
reported  in  the  past  that  it  is  important  to  demonstrate  that  system 
reliability  is  on  track  to  meet  goals  before  production  begins  as  changes 
after  production  commences  can  be  inefficient  and  costly. 


GAO,  Best  Practices:  Capturing  Design  and  Manufacturing  Knowledge  Early  Improves 
Acquisition  Outcomes,  GAO-02-701  (Washington,  D.C.:  July  15,  2002). 
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Testing  Has  Been 
Slow  and  Has  Not 
Demonstrated  That 
the  Aircraft  Will  Work 
in  Its  Intended 
Environment 


The  JSF  program  is  still  very  early  in  demonstrating  aircraft  design  and 
testing  to  verify  it  works  as  intended.  As  of  December  2010,  about  four 
percent  of  JSF  capabilities  have  been  completely  verified  by  flight  tests, 
lab  results,  or  both.  Initial  tests  of  a  fully  integrated  aircraft  to 
demonstrate  full  mission  systems  capabilities  and  weapons  delivery  is  now 
not  expected  until  2015,  three  years  later  than  planned.  The  program 
demonstrated  measurable  progress  in  development  flight  testing  during 
2010,  but  still  lags  earlier  expectations,  and  the  STOVL  problems  have 
constrained  overall  progress.  Only  3  of  32  ground  test  labs  and  simulation 
models  critical  to  complement  and,  in  some  cases,  substitute  for  flight 
tests,  are  accredited  to  verify  and  ensure  the  fidelity  of  results.  Software 
development — essential  for  achieving  about  80  percent  of  the  JSF 
functionality — is  significantly  behind  schedule  as  it  enters  its  most 
challenging  phase.  Software  delivery  to  the  test  program  that  is  essential 
to  demonstrating  full  system  capability  is  now  expected  in  late  2014,  a 
3-year  delay. 


Our  work  in  best  practices  suggests  that  a  key  indicator  of  a  product’s 
maturity  and  readiness  for  production  is  when  a  fully  integrated,  capable 
system  has  been  demonstrated  to  work  in  its  intended  environment.  A 
fully  integrated,  capable  system  would  include  the  integration  of  all  the 
hardware,  including  mission  avionics  systems,  and  software  needed  to 
provide  the  system  its  full  mission  capabilities.  Many  past  DOD  weapons 
programs  have  failed  to  demonstrate  that  the  system  works  as  intended 
before  entering  production,  discovering  costly  design  problems  late  in 
development  when  the  more  complex  software  and  advanced  capabilities 
are  integrated  and  tested. 


Development  Flight  Development  flight  testing  was  much  more  active  in  2010  than  prior  years 

Testing  Is  Progressing  but  had  some  notable  successes,  but  overall  still  lagged  behind 

Behind  Plans  expectations.  The  continuing  effects  from  late  delivery  of  test  aircraft  and 

an  inability  to  achieve  the  plarmed  flying  rates  per  aircraft  substantially 
reduced  the  amount  and  pace  of  testing  planned  previously.  Consequently, 
even  though  the  flight  test  program  accelerated  its  pace  last  year,  the  total 
number  of  flights  accomplished  during  the  first  four  years  of  the  test 
program  significantly  lagged  expectations  when  the  program’s  2007 
baseline  was  established.  Figure  5  shows  that  the  cumulative  number  of 
flights  accomplished  by  the  end  of  2010  was  only  about  one-fifth  the 
number  forecast  by  this  time  in  the  2007  test  plan. 
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Figure  5:  Actual  JSF  Flight  Tests  Completed  through  2010  Compared  to  the  2007 
Plan 
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Source:  GAO  analysis  of  DOD  data. 


Program  officials  reported  that  13  test  aircraft  are  now  out  of  production. 
Ten  test  aircraft  have  been  ferried  to  test  sites  and  others  are  in  varying 
stages  of  final  check-out.  The  program  has  accomplished  first  flights  for  all 
three  variants.  Officials  had  hoped  aircraft  could  achieve  a  rate  of  12 
flights  per  month.  However,  the  average  flight  rate  for  2010  ranged  from 
over  2  to  almost  8  per  month. 

By  the  end  of  2010,  about  10  percent  of  more  than  fifty  thousand  planned 
test  points  have  been  completed.  According  to  program  officials, 
completion  of  a  test  point  means  that  the  test  point  has  been  flown  and 
that  flight  engineers  ruled  that  the  point  has  met  the  need.  Further  analysis 
may  be  necessary  for  the  test  point  to  be  closed  out.  The  majority  of  the 
points  were  earned  on  airworthiness  tests  (basic  airframe  handling 


This  includes  12  test  aircreift  and  the  non-production  representative  model  that  achieved 
much  of  the  test  flights  prior  to  2010.  A  14th  test  aircraft,  the  test  carrier  variemt  added  in 
the  recent  restructuring  is  expected  to  be  delivered  in  2012. 
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characteristics)  and  in  ferrying  the  planes  to  test  sites.  According  to  a 
senior  level  DOD  test  official,  airworthiness  and  ferry  test  points  should  be 
relatively  easy  to  accomplish.  Remaining  test  points  include  more  complex 
and  stringent  requirements,  such  as  mission  systems,  ship  suitability,  and 
weapons  integration  that  have  yet  to  be  demonstrated. 

As  discussed  earlier,  STOVL  flight  performance  lagged  plans  during  2010, 
while  the  CTOL  variant  exceeded  and  the  CV  variant  generally  met  plans. 
Officials  reported  that  design  and  manufacturing  defects  and  excessive 
component  failures  caused  prolonged  maintenance  periods  that  drove  the 
low  fly  rates.  For  instance,  in  the  July  to  August  2010  period,  STOVL  test 
aircraft  were  down  for  unscheduled  maintenance  more  than  half  the  time. 
Further  test  delays  will  likely  cause  the  program  to  miss  critical  future 
milestones.  STOVL  initial  at-sea  testing  will  not  start  until  October  2011 
because  of  delays  in  clearing  the  vertical-landing  envelope.  STOVL-related 
delays  are  also  causing  Marine  Corps  leadership  to  reassess  its 
requirements  and  will  likely  extend  the  date  for  achieving  initial 
operational  capabilities,  currently  set  in  December  2012. 

Concerned  that  STOVL  testing  problems  were  negatively  affecting  the 
other  variants,  the  Department  moved  to  decouple  the  STOVL  testing  and 
placed  the  variant  on  a  two-year  probation  period  to  work  out  problems 
and  get  back  on  track.  The  Secretary’s  actions  will  require  a  new  test  plan 
since  current  flight  test  plans  rely  substantially  on  the  STOVL  to  fly  and 
demonstrate  test  points  in  common  with  other  variants.  The  current  plan 
has  the  STOVL  responsible  for  completing  about  43  percent  of  the  total 
test  points. 
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Figure  6:  Test  Point  Distribution  by  Variant  (2010  Test  Plan) 


Source:  GAO  analysis  ofDoD  data. 


JSF  restructuring  actions  are  positive  and  support  a  more  robust  and 
achievable  test  plan.  Officials  added  more  resources  for  development 
testing,  extended  the  flight  test  schedule,  and  reduced  the  overlap  with 
initial  operational  testing.  More  recently,  officials  revised  the  test  plan 
increasing  the  total  number  of  test  flights  from  5,856  to  7,727,  about  one- 
third  more.  To  increase  capacity,  the  restructure  added  one  carrier  variant 
test  aircraft,  an  additional  software  integration  line,  and  allowed  the 
program  to  utilize  up  to  three  production  aircraft  for  development  testing. 
Compared  to  the  previous  test  plan,  officials  assumed  more  ground  time 
for  aircraft  maintenance  and  planned  modifications,  as  well  as  a  more 
measured  ramp-up  in  the  rate  of  flights  per  test  aircraft.  The  restructuring 
largely  reverses  the  program’s  earlier  Mid-Course  Risk  Reduction  plan  that 
reduced  test  resources.  Our  March  2008  report^**  criticized  DOD’s  mid¬ 
course  plan,  particularly  the  cuts  made  in  flight  test  assets  and  the  number 
of  flight  tests,  as  well  as  the  program’s  failure  to  address  root  causes  of 
cost  growth,  the  very  reasons  why  officials  felt  the  mid-course  plan  was 


GAO,  Joint  Strike  Fighter:  Recent  Decisions  by  DOD  Add  to  Program  Risks, 
GAO-08-388  (Washington,  D.C.:  Mar.  11,  2008). 
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needed.  Since  that  report  was  issued,  JSF  cost  and  schedule  continued  to 
deteriorate  and  officials  recognized  a  need  to  increase  test  assets  and  add 
more  flight  testing. 


Most  Ground  Test  Labs  The  JSF  test  program  relies  much  more  heavily  than  previous  weapon 

and  Simulation  Models  Are  systems  on  its  modeling  and  simulation  labs  to  test  and  verify  aircraft 
Not  Accredited  or  Verified  design  and  subsystem  performance.  However,  only  3  of  32  labs  and 

models  have  been  fully  accredited  to  date;  the  program  had  plarmed  to 
accredit  11  labs  and  models  by  now.  Accreditation  is  essential  to  ensure 
the  fidelity  of  results  validate  that  the  models  accurately  reflect  aircraft 
performance.  Accreditation  is  a  lengthy  and  involved  technical  evaluation 
using  flight  test  data  to  verify  lab  results.  Much  work  remains  before  the 
program  can  fully  utilize  the  models  and  simulation  capabilities  needed  to 
verify  results  and  to  demonstrate  that  ground  testing  can  substitute  for 
flight  testing.  However,  the  ability  to  substitute  is  unproven  and  progress 
in  reducing  program  risk  is  difficult  to  assess.  Contracting  officials  told  us 
that  early  results  are  providing  good  correlation  between  ground  and  flight 
tests. 

The  Director  of  Operational  Test  and  Evaluation  reported  that  50  percent 
of  the  models  will  be  accredited  during  the  final  year  of  flight  testing,  a 
highly  risky  approach.  Delays  in  accreditation  add  risks  to  not  completing 
future  software  blocks  on  time  and  for  discovering  defects  late.  More  flight 
testing  may  be  needed  to  cover  lab  shortcomings,  but  is  generally  more 
expensive,  and  could  lead  to  more  delays  in  completing  development  and 
operational  testing.  It  could  also  require  more  production  aircraft  for  a 
longer  period  to  supplement  test  assets,  resulting  in  fewer  systems  at 
training  sites  and  operational  bases. 

Contractor  utilization  of  labs  has  increased  markedly  and  the  number  and 
integration  of  labs  is  impressive,  but  capacity  may  be  constrained.  Because 
of  development  concurrency,  there  is  overlap  in  scheduling  the  new 
blocks  and  resources  must  be  shared  between  blocks  when  rework  on  an 
earlier  block  is  required.  If  integration  and  test  is  delayed  due  to  capacity 
or  conflict  with  an  earlier  block,  lab  officials  said  that  expected 
capabilities  may  not  be  delivered  on  time  to  meet  flight  test  and  training 
dates.  Mitigating  strategies  include  adding  people,  lab  capacity,  software 
test  lines,  and  shifting  capabilities  to  later  blocks.  The  2010  restructuring 
added  $250  million  to  increase  integration  lab  capacity.  According  to 
program  officials,  the  greater  number  of  labs  allows  engineers  to  work 
simultaneously  on  different  development  blocks,  reducing  bottlenecks 
that  may  occur  in  testing.  Program  and  contractor  officials  believe  that  the 
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up-front  investment  of  $5  billion  in  simulation  labs  will  pay  off  in  early  risk 
reduction,  reduce  flights,  control  costs,  and  are  essential  to  meet  key 
milestones  in  JSF’s  aggressive  test  plan. 


Software  Development  Is  Software  providing  essential  JSF  capability  is  not  mature  and  releases  to 
behind  Schedule  with  Most  program  are  behind  schedule.  Officials  underestimated  the  time 

Complex  Work  Still  Ahead  effort  needed  to  develop  and  integrate  the  software,  substantially 

contributing  to  the  program’s  overall  cost  and  schedule  problems  and 
testing  delays,  while  requiring  the  retention  of  engineers  for  longer 
periods.  Significant  learning  and  development  work  remains  before  the 
program  can  demonstrate  the  mature  software  capabilities  needed  to  meet 
warfighter  requirements.  Good  progress  has  been  made  in  the  writing  of 
software  code — about  three-fourths  of  the  software  has  been  written  and 
integrated,  but  testing  is  behind  schedule  and  the  most  complex  work  is 
still  ahead.  Program  restructuring  added  a  second  software  integration 
line  which  should  improve  throughput. 

The  JSF  software  development  effort  is  one  of  the  largest  and  most 
complex  in  DOD  history,  providing  80  percent  of  JSF’s  functionality 
essential  to  capabilities  such  as  sensor  fusion,  weapons  and  fire  control, 
maintenance  diagnostics,  and  propulsion.  JSF  has  about  8  times  more  on¬ 
board  software  lines  of  code  than  the  F/A-18E/F  Super  Hornet  and  4  times 
more  than  the  F-22A  Raptor.  Also,  the  amount  of  code  needed  will  likely 
increase  as  integration  and  testing  efforts  intensify.  In  2009,  officials 
reported  that  about  40  percent  of  the  software  had  completed  integration 
and  testing.  They  did  not  provide  us  a  progress  report  through  2010. 
Integration  and  test  is  a  lengthy  effort  and  is  typically  the  most  challenging 
phase  of  software  development  requiring  specialized  skills  and  integration 
test  lines.  The  program  has  experienced  a  growth  of  40  percent  in  total 
software  lines  of  code  since  preliminary  design  review  and  13  percent 
growth  since  the  critical  design  review.  Other  recent  defense  acquisitions 
have  experienced  30  to  100  percent  growth  in  software  over  time. 

Software  capabilities  are  developed,  tested,  and  delivered  in  5  blocks,  or 
increments.  Several  blocks  have  grown  in  size  and  taken  longer  to 
complete  than  planned.  Software  defects,  low  productivity,  and 
concurrent  development  of  successive  blocks  created  inefficiencies, 
taking  longer  to  fix  defects  and  delaying  the  demonstration  of  critical 
capabilities.  In  addition,  program  and  prime  contractor  officials 
acknowledge  they  do  not  include  integration  as  a  key  tracking  metric  and 
have  been  unable  to  agree  on  how  to  track  it.  This  has  made  it  hard  for  the 
program  to  analyze  integration  trends  and  take  action  to  remedy  the 
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situation.  Instead  the  program  office  and  prime  contractor  have  made 
several  adjustments  to  the  software  development  schedule,  each  time 
lengthening  the  time  needed  to  complete  work,  as  shown  in  figure  7. 


Figure  7:  Slips  in  Delivering  Software  to  Flight  Test 
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Source:GAO  analysis  ofDOD  data. 


Delays  in  developing,  integrating,  and  releasing  software  to  the  test 
program  have  cascading  effects  hampering  flight  tests,  training,  and  lab 
accreditation.  While  progress  is  being  made,  a  substantial  amount  of 
software  work  remains  before  the  program  can  demonstrate  full 
warfighting  capabihty.  The  program  released  block  0.5  for  flight  test 
nearly  2  years  later  than  planned  in  the  2006  plan,  largely  due  to 
integration  problems.  Each  of  the  remaining  three  blocks — providing  full 
mission  systems  and  warfighting  capabilities — are  now  projected  to  slip 
between  2  to  3  years  compared  to  the  2006  plan.  Defects  and  workload 
bottlenecks  delayed  the  release  of  full  block  1  capabilities;  the  initial 
limited  release  of  block  1  software  was  flown  for  the  first  time  in 
November  2010.  Software  defects  increased  throughout  2010,  but  fixing 
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defects  did  not  keep  pace.  Some  capabilities  were  moved  to  future  blocks 
in  attempts  to  meet  schedule  and  mitigate  risks.  For  example,  full  data 
fusion  mission  systems  “  were  deferred  from  block  2  to  3.  Further  trades 
and  deferrals  may  be  needed.  Rather  than  working  all  blocks  concurrently, 
focusing  efforts  on  a  more  measured  evolutionary  approach  could  result  in 
more  timely  release  of  incremental  capabilities  to  the  testing,  training,  and 
warfighter  communities.  Development  and  integration  of  the  most 
advanced  capabilities  could  be  deferred  to  future  increments  and 
delivered  to  the  warfighter  at  a  later  date. 

The  recent  technical  baseline  review  identified  software  as  a  significant 
challenge,  slowing  system  development  and  requiring  more  time  and 
money.  Although  officials  are  confident  that  such  risks  can  be  addressed, 
the  scale  and  complexity  of  what  is  involved  remains  a  technically 
challenging  and  lengthy  effort.  Uncertainties  pertaining  to  critical 
technologies,  including  the  helmet-mounted  display  and  advanced  data 
links,  add  to  challenges.  Deficiencies  in  the  helmet  mounted  display, 
especially  latency  in  transmitting  sensor  data,  are  causing  officials  to 
develop  a  second  helmet  while  trying  to  fix  the  first  model.  Resolution 
could  result  in  a  major  redesign  or  changes  in  the  JSF’s  concept  of 
operations  by  placing  limitations  on  the  operational  environment, 
according  to  program  officials. 


Conclusions 


The  JSF  program  is  at  a  critical  juncture — 9  years  in  development  and 
4  years  in  limited  production,  but  still  early  in  testing  and  verifying  aircraft 
performance.  If  effectively  implemented  and  sustained,  the  Department’s 
restructuring  should  place  the  JSF  program  on  firmer  footing  and  lead  to 
more  achievable  and  predictable  outcomes.  However,  restructuring  comes 
with  a  price  tag — higher  up-front  development  costs,  fewer  aircraft 
received  in  the  near  term,  training  delays,  and  prolonged  times  for  testing 
and  delivering  the  capabilities  required  by  the  warfighter.  Reducing  near- 
term  procurement  quantities  lessens  concurrency,  but  the  overlap  among 
development,  testing,  and  production  activities  is  still  substantial  and 
risky.  Development  and  testing  activities  will  now  overlap  11  years  of 
production  based  on  the  latest  extension  in  key  milestones.  Flight  testing 


Mission  systems  are  critical  to  realizing  increased  warfighter  capability  in  combat 
effectiveness  through  next  generation  sensors  with  fused  information  from  on-board  and 
off-board  systems  (i.e.  electronic  warfare,  communication  navigation  identification, 
electro-optical  target  system,  electro-optical  distributed  aperture  system,  radar,  and  data 
links). 


Page  31 


GAO-11-325  Joint  Strike  Fighter 


and  production  activity  are  increasing  and  contractors  are  improving 
supply  and  manufacturing  processes,  but  deliveries  are  still  lagging.  The 
challenge  in  front  of  the  aircraft  and  engine  contractors  is  improving  the 
global  supply  chain  and  accelerating  manufacturing  throughput  to  produce 
quality  products  in  economic  quantities  and  on  time.  Slowed  deliveries 
have  built  a  growing  backlog  of  jets  on  order  but  not  delivered;  this  is  not 
a  good  use  of  federal  funds,  tying  up  millions  of  obligated  dollars  much 
ahead  of  the  ability  of  the  manufacturing  process  to  produce.  The 
Secretary  of  Defense  used  similar  reasoning  in  significantly  reducing 
STOVL  procurement  until  technical  issues  are  resolved  and  the 
manufacturing  process  able  to  deliver  efficiently  and  on  time. 

The  JSF  acquisition  demands  an  unprecedented  share  of  the  Department’s 
future  investment  funding.  The  program’s  size  and  priority  is  such  that  its 
cost  overruns  and  extended  schedules  are  either  borne  by  funding  cuts  to 
other  programs  or  else  drive  increases  in  the  top  line  of  defense  spending, 
the  latter  not  an  attractive  option  in  a  period  of  more  austere  budgets.  Up 
until  now,  JSF  problems  have  been  addressed  either  with  more  time  and 
money  or  by  deferring  aircraft  procurement  to  be  borne  by  future  years’ 
budgets.  It  is  past  time  to  place  some  boundaries  on  the  program  such 
that  future  difficulties  can  be  managed  within  a  finite  budget  by  facilitating 
trades  within  the  JSF  program  and  thereby  minimizing  impacts  on  other 
defense  programs  and  priorities.  Also,  Department  actions  to  limit  STOVL 
procurement,  decouple  it  from  development  testing,  and  concentrate 
efforts  to  resolve  deficiencies  are  appropriate.  Given  its  criticality  to  the 
Marine  Corp’s  future  tactical  aviation  plans,  additional  steps  may  be 
needed  to  set  the  framework  and  criteria  for  the  “probation  period”  and  to 
sustain  management  focus  on  STOVL  in  order  to  better  ascertain  its 
progress  and  inform  future  decisions.  Focused  individual  attention  on 
STOVL  apart  from  the  other  two  variants  could  allow  each  variant  to 
proceed  through  development  and  testing  at  its  own  pace.  Furthermore, 
development  testing  is  hampered  both  by  the  late  delivery  of  software 
increments  and  the  lagging  schedule  for  accrediting  ground  labs  and 
simulation  models.  A  comprehensive  independent  review  of  the  software 
development  process  and  lab  accreditation  issues  could  enhance 
management  insight  and  identify  opportunities  for  improvement  in  these 
critical  areas.  We  note  that  the  previous  independent  teams  established  by 
the  Department  significantly  improved  the  manufacturing,  engine,  and  cost 
estimating  processes. 

We  agree  with  defense  leadership  that  a  renewed  and  sustained  focus  on 
affordability  by  contractors  and  the  Government  is  critical  to  moving  this 
important  program  forward  and  enabling  our  military  services  and  our 


Page  32 


GAO-11-325  Joint  Strike  Fighter 


allies  to  acquire  and  sustain  JSF  forces  in  needed  quantities.  Maintaining 
senior  leadership’s  increased  focus  on  program  results,  holding 
government  and  contractors  accountable  for  improving  performance,  and 
bringing  a  more  assertive,  aggressive  management  approach  for  the  JSF  to 
“live  within  its  means”  could  help  effectively  manage  growth  in  the 
program  and  limit  the  consequences  on  other  programs  in  the  portfolio. 
Controlling  JSF  future  cost  growth  would  minimize  funding  disruption  and 
help  stabilize  the  defense  acquisition  portfolio  by  providing  more  certainty 
to  financial  projections  and  by  facilitating  the  allocation  of  remaining 
budget  authority  to  other  defense  modernization  programs. 


Recommendations  for 
Executive  Action 


1.  The  JSF  program  should  maintain  total  annual  funding  levels  for 
development  and  procurement  at  the  current  budgeted  amounts  in  the 
fiscal  year  2012-2016  future  years  defense  plan  (modified,  if  warranted, 
by  the  new  acquisition  program  baseline  expected  this  year).  It  should 
facilitate  trades  among  cost,  schedule,  requirements,  and  quantities  to 
control  cost  growth.  Having  gone  through  the  Technical  Baseline 
Review  (TBR)  and  budget  approval  process,  it  is  reasonable  to  expect 
the  program  to  execute  against  the  future  years  defense  plan  going 
forward.  Only  in  instances  of  major  and  unforeseen  circumstances, 
should  the  Department  consider  spending  more  money  on  the 
program.  Even  then,  we  would  expect  changes  to  be  few  and  adopted 
only  after  close  scrutiny  by  defense  leadership.  Approved  changes 
should  be  well  supported,  adequately  documented,  and  reported  to  the 
congressional  defense  committees. 

2.  Establish  criteria  for  the  STOVL  probation  period  and  take  additional 
steps  to  sustain  individual  attention  on  STOVL-specific  issues, 
including  independent  F-35B/STOVL  Progress  Reviews  with  Senior 
Leadership  to  ensure  cost  and  schedule  milestones  are  achieved  to 
deliver  required  warfighter  capabilities.  The  intent  is  to  allow  each  JSF 


Given  the  other  priorities  that  DOD  must  address  in  a  finite  budget,  a 
renewed  and  sustained  focus  on  affordability  by  contractors  and  the 
Government  is  critical  for  successfully  moving  the  JSF  program  forward. 
DOD  must  plan  ahead  for  a  way  to  address  and  manage  JSF  challenges 
and  risks  in  the  future.  To  facilitate  making  tradeoff  decisions  with 
respect  to  the  JSF  program  that  limit  impacts  to  other  DOD  programs  and 
priorities  and  to  improve  key  management  processes,  we  recommend  that 
the  Secretary  of  Defense  take  the  following  actions  to  reinforce  and 
strengthen  program  cost  controls  and  oversight: 
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variant  to  proceed  and  demonstrate  success  at  its  own  pace  and  could 
result  in  separate  full-rate  production  decisions. 

3.  The  Department  should  conduct  an  independent  review  of  the 

contractor’s  software  development,  integration,  and  test  processes — 
similar  to  its  review  of  manufacturing  operations — and  look  for 
opportunities  to  streamline  software  efforts.  This  review  should 
include  an  evaluation  of  the  ground  lab  and  simulation  model 
accreditation  process  to  ensure  it  is  properly  structured  and  robustly 
resourced  to  support  software  test  and  verification  requirements. 


Agency  Comments 
and  Our  Evaluation 


DOD  provided  us  with  written  comments  on  a  draft  of  this  report.  The 
comments  are  reprinted  in  appendix  111.  We  worked  collaboratively  with 
defense  officials  to  hone  our  draft  recommendations,  making  them  more 
targeted.  DOD  concurred  with  the  recommendations  as  amended.  We  also 
incorporated  technical  comments  as  appropriate. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Defense;  the 
Secretaries  of  the  Air  Force  and  Navy;  the  Commandant  of  the  Marine 
Corps;  and  the  Director  of  the  Office  of  Management  and  Budget.  The 
report  also  is  available  at  no  charge  on  the  GAO  Web  site  at 
http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841  or  sullivanm@gao.gov.  Contact  points  for  our 
Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found  on  the 
last  page  of  this  report.  Staff  members  making  key  contributions  to  this 
report  are  listed  in  appendix  IV. 


Michael  J.  Sullivan 
Director 

Acquisition  and  Sourcing  Management 
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List  of  Congressional  Committees 

The  Honorable  Carl  Levin 
Chairman 

The  Honorable  John  McCain 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Daniel  K.  Inouye 
Chairman 

The  Honorable  Thad  Cochran 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Howard  P.  McKeon 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  C.W.  Bill  Young 
Chairman 

The  Honorable  Norman  D.  Dicks 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I:  Scope  and  Methodology 


To  determine  the  Joint  Strike  Fighter  (JSF)  program’s  progress  in  meeting 
cost,  schedule,  and  performance  goals,  we  received  briefings  by  program 
and  contractor  officials  and  reviewed  financial  management  reports, 
budget  documents,  annual  Selected  Acquisition  Reports,  monthly  status 
reports,  performance  indicators,  and  other  data.  We  identified  changes  in 
cost  and  schedule,  and  obtained  officials’  reasons  for  these  changes.  We 
interviewed  officials  from  the  JSF  program,  contractors,  and  the 
Department  of  Defense  (DOD)  to  obtain  their  views  on  progress,  ongoing 
concerns  and  actions  taken  to  address  them,  and  future  plans  to  complete 
JSF  development  and  accelerate  procurement.  At  the  time  of  our  review, 
the  most  recent  Selected  Acquisition  Report  available  was  dated 
December  31,  2009  and  released  in  April  2010.  At  the  time  of  our  review, 
DOD  was  preparing  a  new  acquisition  program  baseline  for  the  program 
which  would  reflect  updated  cost  and  schedule  projections. 

In  assessing  program  cost  estimates,  we  compared  the  official  program 
cost  estimate  in  the  2009  Selected  Acquisition  Report  and  subsequent  cost 
estimate  developed  after  the  Nunn-McCurdy  breach  to  estimates 
developed  by  the  JSF  program  and  Defense  Contract  Management  Agency 
(DCMA)  reports.  We  interviewed  program  office  officials  and  members  of 
the  DOD  Cost  Analysis  and  Program  Evaluation  Office  (CAPE),  and  DCMA 
to  understand  their  methodology,  data,  and  approach  in  developing  cost 
estimates.  To  assess  the  validity  and  reliability  of  contractors’  cost 
estimates,  we  reviewed  audit  reports  prepared  by  DCMA  and  cost 
performance  reports  prepared  by  the  contractor. 

To  access  the  program’s  plans  and  risk  in  manufacturing  and  its  capacity 
to  accelerate  production,  we  analyzed  manufacturing  cost  and  work 
performance  data  to  assess  progress  against  plans.  We  compared 
budgeted  program  labor  hours  to  actual  labor  hours  and  identified  growth 
trends.  We  reviewed  data  and  briefings  provided  by  the  program,  DCMA, 
and  CAPE  to  assess  supplier  performance  and  ability  to  support 
accelerated  production  in  the  near  term.  We  also  determined  reasons  for 
manufacturing  delays,  discussed  program  and  contractor  plans  to 
improve,  and  projected  the  impact  on  development  and  operational  tests. 
We  interviewed  Naval  Air  Systems  Command  and  contractor  officials  to 
discuss  Earned  Value  Management  System  issues  but  we  did  not  conduct 
any  analysis  since  the  data  was  deemed  unreliable  by  DCMA. 

To  assess  plans,  progress,  and  risks  in  test  activities,  we  examined 
program  documents  and  interviewed  DOD,  program  office,  and  contractor 
officials  about  current  test  plans  and  progress.  To  assess  progress 
towards  test  plans,  we  compared  the  number  of  flight  tests  conducted  as 
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of  December  2010  to  the  original  test  plan  established  in  2007.  We  also 
reviewed  documents  and  interviewed  prime  contractors  about  flight 
testing,  the  integrated  airborne  test  bed,  and  ground  testing.  To  assess  the 
ground  labs  and  test  bed,  we  interviewed  officials  and  toured  the  testing 
labs  at  the  Lockheed  Martin  facilities  in  Fort  Worth,  Texas.  We  also 
reviewed  the  independent  assessments  conducted  by  the  JET  and  NAVAIR 
to  obtain  their  perspective  on  the  program’s  progress  in  test  activities. 

In  performing  our  work,  we  obtained  information  and  interviewed  officials 
from  the  JSF  Joint  Program  office,  Arlington,  Virginia;  Naval  Air  Systems 
Command,  Patuxent  River,  Maryland;  Defense  Contract  Management 
Agency,  Fort  Worth,  Texas;  Lockheed  Martin  Aeronautics,  Fort  Worth, 
Texas;  Defense  Contract  Management  Agency,  Middletown,  Cormecticut; 
and  Pratt  &  Whitney,  Middletown,  Connecticut.  We  also  met  with  and 
obtained  data  from  the  following  offices  of  the  Secretary  of  Defense  in 
Washington,  D.C.;  Director,  Operational  Test  and  Evaluation;  Cost 
Analysis  and  Program  Evaluation  Office;  and  Systems  and  Software 
Engineering.  We  assessed  the  reliability  of  DOD  and  JSF  contractor  data 
by  (1)  performing  electronic  testing  of  required  data  elements, 

(2)  reviewing  existing  information  about  the  data,  and  (3)  interviewing 
agency  officials  knowledgeable  about  the  data.  We  determined  that  the 
data  were  sufficiently  reliable  for  the  purposes  of  this  report.  We 
conducted  this  performance  audit  from  May  2010  to  February  2011  in 
accordance  with  generally  accepted  government  auditing  standards. 

Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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Appendix  II 

:  Prior  GAO  Reports  on  JSF  and 

DOD  Responses  and  Subsequent  Actions 

GAO  report 

Est.  dev.  costs 
dev.  length 
APUC“ 

Key  program  event 

Primary  GAO  message 

DOD  response  and  actions 

2001 

GAO-02-39 

$34.4  Billion 

10  years 
$69  Million 

Start  of  system  development 
and  demonstration  approved. 

Critical  technologies  needed  for 
key  aircraft  performance 
elements  are  not  mature. 

P  rogram  should  delay  start  of 
system  development  until 
critical  technologies  are  mature 
to  acceptable  levels. 

DOD  did  not  delay  start  of 
system  development  and 
demonstration  stating 
technologies  were  at 
acceptable  maturity  levels  and 
will  manage  risks  in 
development. 

2005 

GAO-05-271 

$44.8  Billion 

12  years 
$82  Million 

The  program  undergoes  re¬ 
plan  to  address  higher  than 
expected  design  weight,  which 
added  $7  billion  and  18  months 
to  development  schedule. 

We  recommend  that  the 
program  reduce  risks  and 
establish  executable  business 
case  that  is  knowledge-based 
with  an  evolutionary  acquisition 
strategy. 

DOD  partially  concurred  but 
does  not  adjust  strategy, 
believing  that  their  approach  is 
balanced  between  cost, 
schedule  and  technical  risk. 

2006 

GAO-06-356 

$45.7  Billion 

12  years 
$86  Million 

Program  sets  in  motion  plan  to 
enter  production  in  2007  shortly 
after  first  flight  of  the  non¬ 
production  representative 
aircraft. 

The  program  plans  to  enter 
production  with  less  than  1 
percent  of  testing  complete. 

We  recommend  program  delay 
investing  in  production  until 
flight  testing  shows  thatj  SF 
performs  as  expected. 

DOD  partially  concurred  butdid 
not  delay  start  of  production 
because  they  believe  the  risk 
level  was  appropriate. 

2007 

GAO-07-360 

$44.5  Billion 

12  years 
$104  Million 

Congress  reduced  funding  for 
first  two  low-rate  production 
buys  thereby  slowing  the  ramp 
up  of  production. 

Progress  is  being  made  but 
concerns  remain  about  undue 
overlap  in  testing  and 
production.  We  recommend 
limits  to  annual  production 
quantities  to  24  a  year  until 
flying  quantities  are 
demonstrated. 

DOD  non-concurred  and  felt 
that  the  program  had  an 
acceptable  level  of  concurrency 
and  an  appropriate  acquisition 
strategy. 

2008 

GAO-08-388 

$44.2  Billion 

12  years 
$104  Million 

DOD  implemented  a  Mid- 
Course  Risk  Reduction  Plan  to 
replenish  management 
reserves  from  about  $400 
million  to  about  $1  billion  by 
reducing  test  resources. 

We  believe  new  plan  actually 
increases  risks  and  that  DOD 
should  revise  the  plan  to 
address  concerns  about 
testing,  use  of  management 
reserves,  and  manufacturing. 

We  determine  that  the  cost 
estimate  is  not  reliable  and  that 
a  new  cost  estimate  and 
schedule  risk  assessment  is 
needed. 

DOD  did  not  revise  risk  plan 
nor  restore  testing  resources, 
stating  that  they  will  monitor  the 
new  plan  and  adjust  it  if 
necessary.  Consistent  with  a 
report  recommendation,  a  new 
cost  estimate  was  eventually 
prepared,  but  DOD  refused  to 
do  a  risk  and  uncertainty 
analysis  that  we  felt  was 
important  to  provide  a  range 
estimate  of  potential  outcomes. 
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DOD  Responses  and  Subsequent  Actions 


GAO  report 

Est.  dev.  costs 
dev.  length 
APUC“ 

Key  program  event 

Primary  GAO  message 

DOD  response  and  actions 

2009 

GAO-09-303 

$44.4  Biiiion 

13  years 
$104  Miiiion 

The  program  increased  the 
cost  estimate  and  adds  a  year 
to  deveiopment  but  acceierated 
the  production  ramp  up. 
independent  DOD  cost 
estimate  (J  ET  i)  projects  even 
higher  costs  and  further  deiays. 

Because  of  deveiopment 
probiems,  we  stated  that 
moving  forward  with  an 
acceierated  procurement  pian 
and  use  of  cost  reimbursement 
contracts  is  very  risky.  We 
recommended  the  program 
report  on  the  risks  and 
mitigation  strategy  for  this 
approach. 

DOD  agreed  to  report  its 
contracting  strategy  and  pians 
to  Congress,  in  response  to  our 
report  recommendation,  DOD 
subsequentiy  agreed  to  do  a 
scheduie  risk  anaiysis,  but  stiii 
had  not  done  so  as  of  February 
2011.  in  February  2010,  the 
Department  announced  a 
major  restructuring  of  the  j  S  F 
program,  inciuding  reduced 
procurement  and  a  pianned 
move  to  fixed-price  contracts. 

2010 

GAO-10-382 

$49.3  Biiiion 

15  years 
$112  Miiiion 

The  program  was  restructured 
to  reflect  findings  of  recent 
independent  cost  team  (J  ET  ii) 
and  independent 
manufacturing  review  team.  As 
a  resuit,  deveiopment  funds 
increased,  test  aircraft  were 
added,  the  scheduie  was 
extended,  and  the  eariy 
production  rate  decreased. 

Because  of  additionai  costs 
and  scheduie  deiays,  the 
program's  abiiity  to  meet 
warfighter  requirements  on 
time  is  at  risk.  We  recommend 
the  program  compiete  a  fuii 
comprehensive  cost  estimate 
and  assess  warfighter  and  iOC 
requirements.  We  suggest  that 
Congress  require  DOD  to 
prepare  a  "system  maturity 
matrix"-a  tooi  for  tying  annuai 
procurement  requests  to 
demonstrated  progress. 

DOD  continued  restructuring 
actions  and  announced  pians 
to  increase  test  resources  and 
iowerthe  production  rate, 
independent  review  teams 
evaiuated  aircraft  and  engine 
manufacturing  processes.  As 
we  projected  in  this  report,  cost 
increases  iater  resuited  in  a 
Nunn-McCurdy  breach.  Miiitary 
services  are  currentiy  reviewing 
capabiiity  requirements  as  we 
recommended.  The 

Department  and  Congress  are 

working  on  a  "system  maturity 
matrix"  tooi,  which  we 
suggested  to  Congress  for 
consideration,  to  improve 
oversight  and  inform  budget 
deiiberations. 


Source:  DOD  data  and  GAO  analysis. 

Average  procurement  unit  cost. 
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Department  of  Defense 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3000 


ACQUISITION. 
Tfc'CHNOLCGY 
ANO  LOOJSTIGS 


APB  -5  20\] 


Mr.  Michael  Sullivan 

Director.  Acquisition  and  Sourcing  Management 
U.S.  Government  .Accountability  Office 
44 1  G  Street.  NW 
Washington,  DC  20548 

Dear  Mr.  Sullivan: 

I  his  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  Draft  Report, 

G.AO-1 1-325.  "JOrNl  STRIKE  FIGHT  F.R:  Restructuring  Places  Program  on  Firmer  Footing, 
but  Progress  is  Still  Lagging  Overall,"  dated  February  28,  201 1  (GAO  Code  120918).  Detailed 
comments  on  the  report  recommendations  are  enclosed. 

The  DoD  concurs  with  all  three  recommendations.  The  rationale  and  actions  taken  by 
DoD  are  included  in  the  enclosure. 

W'c  appreciate  the  opportunity  to  comment  on  the  draft  report.  My  point  of  contact  is 
Colonel  Jason  Denney,  U.S.  Air  Force,  Jason.Dennev@osd.mil.  703-697-3619. 

Sincerely. 


DaVid  G,  Ahern 

Deputy  Assistant  Secretary  of  Defense 
Portfolio  Systems  Acquisition 


Enclosure: 
As  stated 
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GAO  DRAFT  REPORT  DATED  FEBRUARY  28, 2011 
GAO-11-32S  (GAO  CODE  120918) 

“JOINT  STRIKE  FIGHTER:  RESTRUCTURING  PLACES 

PROGRAM  ON  FIRMER  FOOTING,  BUT  PROGRESS  IS  STILL 
LAGGING  OVERALL” 

The  Department  remains  committed  to  the  F-35  Joint  Strike  Fighter  (JSF) 
program.  The  Fiscal  Year  (FY)  2012  President’s  Budget  (PB)  demonstrates  this 
commitment  to  the  F-35  as  the  backbone  of  the  future  tactical  aircraft  inventory  for  the 
Air  Force,  Navy,  Marine  Corps,  as  well  as  our  Intcmational  Partners. 

The  Department  and  the  F-35  prime  contractor  are  committed  to  delivering  F-35 
aircraft  that  meet  the  Services’  requirements  as  specified  in  the  Joint  Operational 
Requirements  Document  (JORD)  in  addition  to  controlling  and  reducing  costs  wherever 
and  whenever  possible,  with  the  goal  of  providing  the  Services’  an  affordable  tactical 
aviation  capability. 

Following  a  Critical  Nunn-McCurdy  Breach,  a  rescission  of  the  original 
October  26, 2001  Milestone  B  (MS  B)  decision,  and  program  recertification  in  June 
2010,  the  Department  set  out  to  establish  a  firmer  foundation  for  the  program.  The  F-35 
Program  Executive  Officer  (PEO)  executed  an  extensive  hottoms-up  Technical  Baseline 
Review  (TBR)  involving  over  120  tactical  aircraft  experts  from  both  the  Services  and  the 
Department  to  evaluate  every  aspect  of  the  System  Development  and  Demonstration 
(SDD)  phase  of  the  program.  In  addition  to  the  TBR,  the  PEO  will  execute  a  schedule 
risk  assessment  this  summer  and  finalize  the  program’s  Integrated  Master  Schedule  this 
fall  following  an  Integrated  Baseline  Review. 

The  Department  will  hold  a  Defense  Acquisition  Board  (DAB)  in  May  201 1  to 
review  the  restructured  SDD  phase  of  the  program  and  update  the  Acquisition  Program 
Baseline. 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 

RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defease  to 
maintain  total  annual  funding  levels  for  development  and  procurement  at  the  current 
budgeted  amounts  in  the  fiscal  year  2012-2016  future  years  defense  plan  (modified,  if 
warranted,  by  the  new  acquisition  program  baseline  expected  this  year).  It  should 
facilitate  trades  among  cost,  schedule,  requirements,  and  quantities  to  control  cost 
growth.  Having  gone  through  the  Technical  Baseline  Review  (TBR)  review  and  budget 
approval  process,  it  is  reasonable  to  expect  the  program  to  execute  against  the  FYDP 
budget  going  forward.  Only  in  instances  of  major  and  unforeseen  circumstances,  should 
the  Department  consider  spending  more  money  on  the  program.  Even  then,  we  would 
expect  changes  to  be  few  and  adopted  only  after  close  scrutiny  by  defense  leadership. 
Approved  changes  should  be  well  supported,  adequately  documented,  and  reported  to  the 
congressional  defense  committees. 
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POD  RESPONSE:  Concur  The  Department  is  confident  that  the  F-35  budget  request 
for  FY  2012-2016  provides  the  appropriate  level  of  funding  for  the  restructured 
development  program  and  revised  procurement  profile.  The  Department  undertakes  a 
very  thorough  and  in-depth  process  to  develop  the  annual  Defense  Budget.  The  goal  is  to 
prepare  a  budget  that  does  not  change  significantly  fi'om  year  to  year  and  provides 
acquisition  programs  and  the  Services  a  stable  fundir^  and  procurement  profile  to  plan 
against.  In  this  instance,  the  JSF  Program  Office  conducted  an  extensive  TBR  involving 
over  120  tactical  aircraft  experts  ftom  both  the  Services  and  the  Department  to  evaluate 
every  aspect  of  the  SDD  phase  of  the  program.  The  resultant  schedule  and  cost 
adjustments  made  following  the  TBR,  and  reflected  in  the  FY2012  Budget,  afforded  the 
Department  a  more  realistic  basis  to  manage  the  program.  The  approval  of  a  new  MS  B 
and  APB  will  reflect  these  budget  decisions.  Any  significant  proposed  changes  to  the 
F-35  budget  in  future  years  will  be  thoroughly  reviewed  as  part  of  the  annual  Defense 
Acquisition  Executive  Program  Reviews  and  approved  as  part  of  the  budget  review  and 
submission  process. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  to 
establish  criteria  for  the  short  take  off  and  landing  variant  (STOVE)  probation  period  and 
take  additional  steps  to  sustain  individual  attention  to  STOVL-specifle  issues,  including 
independent  F-35B/STOVL  Progress  Reviews  with  Senior  Leadership  to  ensure  cost  and 
schedule  milestones  are  achieved  to  deliver  required  warfighter  capabilities.  The  intent  is 
to  allow  each  JSF  variant  to  proceed  and  demonstrate  success  at  its  own  pace  and  could 
result  in  separate  full-rate  production  decisions. 

POD  RESPONSE:  Concur.  The  Service  Acquisition  Executives  (SAEs)  for  the 
Department  of  the  Navy  and  Air  Force  have  established  a  battle  rhythm  of  monthly  SAE 
reviews  with  the  F-35  PEG  to  assess  the  overall  F-35  program,  with  additional  emphasis 
on  F-35B  Short  Take  Off  and  Vertical  Landing  (STOVL)  variant.  These  monthly  SAE 
Reviews  examine  the  F-35  airframe  and  propulsion  SDD,  Production,  and  Sustainment 
programs  with  particular  emphasis  on  Cost/Affordability,  Risk,  and  Schedule.  In 
addition,  the  Commandant  of  the  Marine  Corps  has  established  a  monthly  F-35  review 
focused  solely  on  the  F-35B  variant.  This  review  is  also  led  by  Department  of  the  Navy 
(DoN)  Senior  Acquisition  Executives.  The  topics  and  metrics  that  are  assessed  include, 
but  are  not  limited  to,  the  following: 

•  Cost/Affordability/Eamed  Value  Management  (EVM);  A  review  of  Acquisition 
Procurement  Unit  Cost,  Program  Acquisition  Unit  Cost,  Operations  and  Support 
costs,  and  EVM  cost/schedule  indices. 

•  Risk;  Monthly  assessment  of  ‘Program  Risk’  with  explanations  about  each  risk 
item,  their  interactions,  and  risk  bum-down  plans.  Assessments  include  a  review 
of  the  assumptions  and  environment  used  to  determine  the  risk  evaluations. 

•  F-35B  weight/weight  growth:  Weight  assessments  track  each  pound  added  to  the 
airframe  with  an  understanding  of  the  underlying  reasons  for  the  growth.  If  there 
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are  trades  that  need  to  be  made  to  mitigate  weight  growth,  DoN  senior 
leadership/warfightets  are  to  be  consulted. 

•  Key  Performance  Parameters  (KPPs):  Review  of  F-35B  KPPs  with  a  tracking/ 
trending  methodology  and  monthly  discussions  of  CONORS  considerations  that 
might  help  facilitate  achievement  of  F-35  KPPs  and  program  goals. 

•  Airframe  Technical  Performance  Measurements:  Assessments  of  reliability, 
maintainability,  combat  radius,  and  gross  weight  (with  metrics  that  indicate  the 
desired  value(s),  the  current  status,  margin,  and  trends). 

•  F-35B  Flight  Test;  Review  of  F-35B  flight  test  data,  to  include, 
planned/scheduled  test  points  to  be  flown  versus  achieved  test  points  flown; 
scheduled  test  flights  flown  versus  actual  test  flights  flown  (delineated  by  STOVL 
variant);  and  Clean- Wing  Flight  Envelope  coverage  (to  assess  the  progress  on  the 
envelope  cleared  for  flight  as  a  result  of  Developmental  Test  and  alignment  with 
software  delivery). 

As  these  monthly  reviews  mature,  the  DoN  will  refine  key  F-35B  metrics  to  ensure  this 
essential  capability  is  delivered  to  the  Marine  Corps  Warfighters  and  to  enable  the 
Department  to  make  a  decision  on  the  F-35B  STOVL  variant  probation  status. 

RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  to 
conduct  an  independent  review  of  the  contractor’s  software  development,  integration,  and 
test  processes — similar  to  its  review  of  manufacturing  operations — and  look  for 
opportunities  to  streamline  software  efforts.  This  review  should  include  an  evaluation  of 
the  ground  lab  and  simulation  model  accreditation  process  to  ensure  it  is  properly 
structured  and  robustly  resourced  to  support  software  test  and  verification  requirements. 

POD  RESPONSE:  Concur  The  Department  believes  that  the  newly  structured 
program,  which  delivers  a  JORD-compliant  Block  3  capable  aircraft  at  the  end  of  the 
SDD  phase,  provides  the  Warfighter  capability  needed.  The  Department  acknowledges 
that  delivery  of  that  capability  has  taken  longer  and  will  cost  more  than  planned,  in  part 
due  to  poor  performance  to  (iite  in  software  development.  The  recent  TBR  ensured  that 
full  Block  3  capability  was  adequately  costed  and  scheduled  with  the  appropriate  amount 
of  acceptable  risk,  to  include  the  risk  in  the  software  development  portion  of  the  program. 
In  order  to  maximize  our  developmental  mvestment,  it  would  be  prudent  to 
independently  verify  the  contractor  has  appropriate  processes  to  develop  and  field 
software  and  is  diligently  following  them  accordingly. 
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